Doorbell camera with battery at chime

ABSTRACT

Doorbell camera systems are provided to include a doorbell camera subsystem and a chime subsystem both coupled to receive power from an alternating current power source. The doorbell camera subsystem may include, among other features, a camera module and a doorbell button. The chime subsystem may include, among other features, a current compensation network and chime driver circuitry operative to be coupled to a chime. The current compensation network is operative to dynamically adjust current consumption of the chime subsystem and current consumption of the doorbell camera subsystem during both a standby mode and a doorbell event mode.

RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 15/813,644, filed Nov. 15, 2017, titled “DoorbellCamera with Battery at Chime,” now U.S. Pat. No. 10,368,040, which isincorporated by reference herein in its entirety.

TECHNICAL FIELD

This relates generally to electronic devices, including but not limitedto systems for mechanically supporting an electronic device thatintegrates a camera module with a doorbell, and that uses a battery thatis located proximal to a doorbell chime.

BACKGROUND

A smart home environment is created at a venue by integrating aplurality of smart devices, including intelligent, multi-sensing,network-connected electronic devices, seamlessly with each other in alocal area network and/or with a central server or a cloud-computingsystem to provide a variety of useful smart home functions.Network-connected video surveillance cameras have been extensively usedin the smart home environment to provide video monitoring and security.Such extensive usage of video cameras in residential and commercialenvironments has increased substantially, in part due to lower pricesand simplicity of deployment.

Sometimes, one or more of the smart devices are located in an outdoorenvironment (e.g., in a porch or a backyard of a house). For example,one or more network-connected cameras are often installed on an outerwall of a house, and configured to provide video monitoring and securityin the outdoor environment. These smart devices (e.g., thenetwork-connected outdoor cameras) are exposed to severe weatherconditions (e.g., a rainfall, a snowstorm and direct sun exposure), andrequire additional power supplies being physically routed to them eventhough these smart device normally can communicate data with a remoteserver or a client device wirelessly via one or more communicationnetworks. Each outdoor smart device must be configured to attach firmlyto a surface in the outdoor environment, have an access to a powersupply source, function reliably under various severe weather conditions(e.g., water intrusion from a rainfall or snowstorm) that could happen,and last for a long duration in the outdoor environment.

SUMMARY

A doorbell camera system that includes a doorbell camera subsystem and achime subsystem is provided. The doorbell camera system can supplyuninterrupted power to the doorbell camera subsystem and the chimesubsystem. This can be accomplished by using a current balancingarchitecture that co-locates a battery with the chime subsystem and notwith the camera doorbell subsystem. In this architecture, the doorbellcamera subsystem is always powered by line power and the chime of thechime subsystem is activated by a battery that is recharged using linepower in between doorbell button presses. Moving the battery away fromdoorbell camera subsystem can enable enhanced design flexibility for thedoorbell camera subsystem. For example, the industrial design doorbellcamera body can be shrunk (i.e., because no battery is present), andadditional features (e.g., improved camera sensor, less featurethrottling, improved speaker quality, and wireless communications withother devices) can be added because the doorbell camera system is nolonger reliant on a battery. Moreover, the by moving the battery awayfrom the doorbell camera subsystem, the battery itself is no exposed topossible high temperature fluctuations outside. In addition, the batterycan be sized bigger because it is no longer confined to the sizelimitations of a doorbell housing.

In one embodiment, a doorbell camera system is provided that includes acamera doorbell subsystem coupled to receive power from an AC powersource, the camera doorbell subsystem comprising: a doorbell button, acamera module, an LED indicator, and a first processor. The doorbellcamera system also includes a chime subsystem coupled to receive powerfrom the AC power source, the chime subsystem comprising: a currentcompensation network, a second processor, a battery, and chime drivercircuity operative to be coupled to a chime.

In another embodiment, a doorbell camera system is provided thatincludes a camera doorbell subsystem coupled to receive power from an ACpower source and a chime subsystem coupled to receive power from the ACpower source. The camera doorbell subsystem can include first powerconditioning circuitry coupled to receive the AC power source, adoorbell button, a camera module coupled to receive power from the firstpower conditioning circuitry, and a first processor coupled to receivepower from the first power conditioning circuitry. The chime subsystemcan include second power conditioning circuitry coupled to receive theAC power source, a current compensation network coupled to the secondpower conditioning circuitry, a second processor coupled to receivepower from the second power conditioning circuitry, battery chargingcircuitry coupled to receive power from the second power conditioningcircuitry, a battery coupled to the battery charging circuitry, andchime driver circuity coupled to the battery charging circuitry, whereinthe chime driver circuitry is operative to activate a chime. The secondprocessor is operative to activate the chime driver circuitry inresponse to a button press event of the doorbell button to activate thechime to produce a sound, wherein the chime driver circuitry uses powersupplied by the battery to activate the chime.

In yet another embodiment, a doorbell camera system is provided thatincludes a AC power source connection node, a camera doorbell subsystemcoupled to the AC power source connection node, camera doorbellsubsystem comprising a camera module and a doorbell button, and a chimesubsystem coupled to the AC power source connection node, the chimesubsystem comprising a battery and a chime, wherein the battery suppliespower to the chime when the chime is activated. The camera doorbellsubsystem is operative to instruct the chime subsystem to activate thechime in response to a button press of the doorbell button.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of the embodimentsdiscussed herein may be realized by reference to the remaining portionsof the specification and the drawings.

FIG. 1 is an example smart home environment in accordance with someimplementations.

FIG. 2A is a block diagram illustrating a representative networkarchitecture that includes a smart home network in accordance with someimplementations.

FIG. 2B is a representative operating environment in which a serversystem interacts with client devices and smart devices in accordancewith some implementations.

FIG. 3A is a block diagram illustrating a representative server system,in accordance with some implementations.

FIG. 3B illustrates various data structures used by someimplementations.

FIG. 4 is a block diagram illustrating a representative smart device,e.g., a doorbell camera, in accordance with some implementations.

FIG. 5 illustrates a representative system architecture for videoanalysis and categorization, in accordance with some implementations.

FIG. 6 is a block diagram illustrating a representative client device,in accordance with some implementations.

FIGS. 7A and 7B are a front view and a rear view of a doorbell camera inaccordance with some implementations, respectively.

FIG. 8 is a schematic block diagram of a doorbell camera system inaccordance with some implementations.

FIG. 9 shows an illustrative current diagram of a doorbell system inaccordance with some implementations.

FIG. 10 shows an illustrative schematic diagram of doorbell system inaccordance with some implementations.

FIG. 11 shows an illustrative timing diagram showing current consumptionof various components within a doorbell system according to anembodiment.

FIG. 12 shows an illustrative current compensation network embodimentthat can be used in connection with a doorbell system according to anembodiment.

FIG. 13 shows an illustrative current compensation network embodimentthat can be used in connection with a doorbell system according to anembodiment.

FIG. 14 shows an illustrative current compensation network embodimentthat can be used in connection with a doorbell system according to anembodiment.

Like reference numerals refer to corresponding parts throughout theseveral views of the drawings.

DESCRIPTION OF IMPLEMENTATIONS

In the following detailed description, for purposes of explanation,numerous specific details are set forth to provide a thoroughunderstanding of the various embodiments. Those of ordinary skill in theart will realize that these various embodiments are illustrative onlyand are not intended to be limiting in any way. Other embodiments willreadily suggest themselves to such skilled persons having the benefit ofthis disclosure.

In addition, for clarity purposes, not all of the routine features ofthe embodiments described herein are shown or described. One of ordinaryskill in the art would readily appreciate that in the development of anysuch actual embodiment, numerous embodiment-specific decisions may berequired to achieve specific design objectives. These design objectiveswill vary from one embodiment to another and from one developer toanother. Moreover, it will be appreciated that such a development effortmight be complex and time-consuming but would nevertheless be a routineengineering undertaking for those of ordinary skill in the art havingthe benefit of this disclosure.

FIG. 1 is an example smart home environment 100 in accordance with someimplementations. The smart home environment 100 includes a structure 150(e.g., a house, office building, garage, or mobile home) with variousintegrated devices. It will be appreciated that devices may also beintegrated into a smart home environment 100 that does not include anentire structure 150, such as an apartment, condominium, or officespace. Further, the smart home environment 100 may control and/or becoupled to devices outside of the actual structure 150. Indeed, severaldevices in the smart home environment 100 need not be physically withinthe structure 150. For example, a device controlling a pool heater 114or irrigation system 116 may be located outside of the structure 150.

It is to be appreciated that “smart home environments” may refer tosmart environments for homes such as a single-family house, but thescope of the present teachings is not so limited. The present teachingsare also applicable, without limitation, to duplexes, townhomes,multi-unit apartment buildings, hotels, retail stores, office buildings,industrial buildings, and more generally any living space or work space.

It is also to be appreciated that while the terms user, customer,installer, homeowner, occupant, guest, tenant, landlord, repair person,and the like may be used to refer to the person or persons acting in thecontext of some particularly situations described herein, thesereferences do not limit the scope of the present teachings with respectto the person or persons who are performing such actions. Thus, forexample, the terms user, customer, purchaser, installer, subscriber, andhomeowner may often refer to the same person in the case of asingle-family residential dwelling, because the head of the household isoften the person who makes the purchasing decision, buys the unit, andinstalls and configures the unit, and is also one of the users of theunit. However, in other scenarios, such as a landlord-tenantenvironment, the customer may be the landlord with respect to purchasingthe unit, the installer may be a local apartment supervisor, a firstuser may be the tenant, and a second user may again be the landlord withrespect to remote control functionality. Importantly, while the identityof the person performing the action may be germane to a particularadvantage provided by one or more of the implementations, such identityshould not be construed in the descriptions that follow as necessarilylimiting the scope of the present teachings to those particularindividuals having those particular identities.

The depicted structure 150 includes a plurality of rooms 152, separatedat least partly from each other via walls 154. The walls 154 may includeinterior walls or exterior walls. Each room may further include a floor156 and a ceiling 158. Devices may be mounted on, integrated with and/orsupported by a wall 154, floor 156 or ceiling 158.

In some implementations, the integrated devices of the smart homeenvironment 100 include intelligent, multi-sensing, network-connecteddevices that integrate seamlessly with each other in a smart homenetwork (e.g., 202 FIG. 2A) and/or with a central server or acloud-computing system to provide a variety of useful smart homefunctions. The smart home environment 100 may include one or moreintelligent, multi-sensing, network-connected thermostats 102(hereinafter referred to as “smart thermostats 102”), one or moreintelligent, network-connected, multi-sensing hazard detection units 104(hereinafter referred to as “smart hazard detectors 104”), one or moreintelligent, multi-sensing, network-connected entryway interface devices106 and 120 (hereinafter referred to as “smart doorbells 106” and “smartdoor locks 120”), and one or more intelligent, multi-sensing,network-connected alarm systems 122 (hereinafter referred to as “smartalarm systems 122”).

In some implementations, the one or more smart thermostats 102 detectambient climate characteristics (e.g., temperature and/or humidity) andcontrol a HVAC system 103 accordingly. For example, a respective smartthermostat 102 includes an ambient temperature sensor.

The one or more smart hazard detectors 104 may include thermal radiationsensors directed at respective heat sources (e.g., a stove, oven, otherappliances, a fireplace, etc.). For example, a smart hazard detector 104in a kitchen 153 includes a thermal radiation sensor directed at astove/oven 112. A thermal radiation sensor may determine the temperatureof the respective heat source (or a portion thereof) at which it isdirected and may provide corresponding blackbody radiation data asoutput.

The smart doorbell 106 and/or the smart door lock 120 may detect aperson's approach to or departure from a location (e.g., an outer door),control doorbell/door locking functionality (e.g., receive user inputsfrom a portable electronic device 166-1 to actuate bolt of the smartdoor lock 120), announce a person's approach or departure via audio orvisual means, and/or control settings on a security system (e.g., toactivate or deactivate the security system when occupants go and come).In some implementations, the smart doorbell 106 includes some or all ofthe components and features of the camera 118. In some implementations,the smart doorbell 106 includes a camera 118, and therefore, is alsocalled “doorbell camera 106” in this application.

The smart alarm system 122 may detect the presence of an individualwithin close proximity (e.g., using built-in IR sensors), sound an alarm(e.g., through a built-in speaker, or by sending commands to one or moreexternal speakers), and send notifications to entities or userswithin/outside of the smart home network 100. In some implementations,the smart alarm system 122 also includes one or more input devices orsensors (e.g., keypad, biometric scanner, NFC transceiver, microphone)for verifying the identity of a user, and one or more output devices(e.g., display, speaker). In some implementations, the smart alarmsystem 122 may also be set to an “armed” mode, such that detection of atrigger condition or event causes the alarm to be sounded unless adisarming action is performed.

In some implementations, the smart home environment 100 includes one ormore intelligent, multi-sensing, network-connected wall switches 108(hereinafter referred to as “smart wall switches 108”), along with oneor more intelligent, multi-sensing, network-connected wall pluginterfaces 110 (hereinafter referred to as “smart wall plugs 110”). Thesmart wall switches 108 may detect ambient lighting conditions, detectroom-occupancy states, and control a power and/or dim state of one ormore lights. In some instances, smart wall switches 108 may also controla power state or speed of a fan, such as a ceiling fan. The smart wallplugs 110 may detect occupancy of a room or enclosure and control supplyof power to one or more wall plugs (e.g., such that power is notsupplied to the plug if nobody is at home).

In some implementations, the smart home environment 100 of FIG. 1includes a plurality of intelligent, multi-sensing, network-connectedappliances 112 (hereinafter referred to as “smart appliances 112”), suchas refrigerators, stoves, ovens, televisions, washers, dryers, lights,stereos, intercom systems, garage-door openers, floor fans, ceilingfans, wall air conditioners, pool heaters, irrigation systems, securitysystems, space heaters, window AC units, motorized duct vents, and soforth. In some implementations, when plugged in, an appliance mayannounce itself to the smart home network, such as by indicating whattype of appliance it is, and it may automatically integrate with thecontrols of the smart home. Such communication by the appliance to thesmart home may be facilitated by either a wired or wirelesscommunication protocol. The smart home may also include a variety ofnon-communicating legacy appliances 140, such as old conventionalwasher/dryers, refrigerators, and the like, which may be controlled bysmart wall plugs 110. The smart home environment 100 may further includea variety of partially communicating legacy appliances 142, such asinfrared (“IR”) controlled wall air conditioners or other IR-controlleddevices, which may be controlled by IR signals provided by the smarthazard detectors 104 or the smart wall switches 108.

In some implementations, the smart home environment 100 includes one ormore network-connected cameras 118 that are configured to provide videomonitoring and security in the smart home environment 100. The cameras118 may be used to determine occupancy of the structure 150 and/orparticular rooms 152 in the structure 150, and thus may act as occupancysensors. For example, video captured by the cameras 118 may be processedto identify the presence of an occupant in the structure 150 (e.g., in aparticular room 152). Specific individuals may be identified based, forexample, on their appearance (e.g., height, face) and/or movement (e.g.,their walk/gait). Cameras 118 may additionally include one or moresensors (e.g., IR sensors, motion detectors), input devices (e.g.,microphone for capturing audio), and output devices (e.g., speaker foroutputting audio). In some implementations, the cameras 118 are eachconfigured to operate in a day mode and in a low-light mode (e.g., anight mode). In some implementations, the cameras 118 each include oneor more IR illuminators for providing illumination while the camera isoperating in the low-light mode. In some implementations, the cameras118 include one or more outdoor cameras. In some implementations, theoutdoor cameras include additional features and/or components such asweatherproofing and/or solar ray compensation.

The smart home environment 100 may additionally or alternatively includeone or more other occupancy sensors (e.g., the smart doorbell 106, smartdoor locks 120, touch screens, IR sensors, microphones, ambient lightsensors, motion detectors, smart nightlights 170, etc.). In someimplementations, the smart home environment 100 includes radio-frequencyidentification (RFID) readers (e.g., in each room 152 or a portionthereof) that determine occupancy based on RFID tags located on orembedded in occupants. For example, RFID readers may be integrated intothe smart hazard detectors 104.

The smart home environment 100 may also include communication withdevices outside of the physical home but within a proximate geographicalrange of the home. For example, the smart home environment 100 mayinclude a pool heater monitor 114 that communicates a current pooltemperature to other devices within the smart home environment 100and/or receives commands for controlling the pool temperature.Similarly, the smart home environment 100 may include an irrigationmonitor 116 that communicates information regarding, irrigation systemswithin the smart home environment 100 and/or receives controlinformation for controlling such irrigation systems.

By virtue of network connectivity, one or more of the smart home devicesof Figure I may further allow a user to interact with the device even ifthe user is not proximate to the device. For example, a user maycommunicate with a device using a computer (e.g., a desktop computer,laptop computer, or tablet) or other portable electronic device 166(e.g., a mobile phone, such as a smart phone). A webpage or applicationmay be configured to receive communications from the user and controlthe device based on the communications and/or to present informationabout the device's operation to the user. For example, the user may viewa current set point temperature for a device (e.g., a stove) and adjustit using a computer. The user may be in the structure during this remotecommunication or outside the structure.

As discussed above, users may control smart devices in the smart homeenvironment 100 using a network-connected computer or portableelectronic device 166. In some examples, some or all of the occupants(e.g., individuals who live in the home) may register their device 166with the smart home environment 100. Such registration may be made at acentral server to authenticate the occupant and/or the device as beingassociated with the home and to give permission to the occupant to usethe device to control the smart devices in the home. An occupant may usetheir registered device 166 to remotely control the smart devices of thehome, such as when the occupant is at work or on vacation. The occupantmay also use their registered device to control the smart devices whenthe occupant is actually located inside the home, such as when theoccupant is sitting on a couch inside the home. It should be appreciatedthat instead of or in addition to registering devices 166, the smarthome environment 100 may make inferences about which individuals live inthe home and are therefore occupants and which devices 166 areassociated with those individuals. As such, the smart home environmentmay “learn” who is an occupant and permit the devices 166 associatedwith those individuals to control the smart devices of the home.

In some implementations, in addition to containing processing andsensing capabilities, devices 102, 104, 106, 108, 110, 112, 114, 116,118, 120, and/or 122 (collectively referred to as “the smart devices”)are capable of data communications and information sharing with othersmart devices, a central server or cloud-computing system, and/or otherdevices that are network-connected. Data communications may be carriedout using any of a variety of custom or standard wireless protocols(e.g., IEEE 802.15.4, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart,ISA100.5A, WirelessHART, MiWi, etc.) and/or any of a variety of customor standard wired protocols (e.g., Ethernet, HomePlug, etc.), or anyother suitable communication protocol, including communication protocolsnot yet developed as of the filing date of this document.

In some implementations, the smart devices serve as wireless or wiredrepeaters. In some implementations, a first one of the smart devicescommunicates with a second one of the smart devices via a wirelessrouter. The smart devices may further communicate with each other via aconnection (e.g., network interface 160) to a network, such as theInternet 162. Through the Internet 162, the smart devices maycommunicate with a server system 164 (also called a central serversystem and/or a cloud-computing system herein). The server system 164may be associated with a manufacturer, support entity, or serviceprovider associated with the smart device(s). In some implementations, auser is able to contact customer support using a smart device itselfrather than needing to use other communication means, such as atelephone or Internet-connected computer. In some implementations,software updates are automatically sent from the server system 164 tosmart devices (e.g., when available, when purchased, or at routineintervals).

In some implementations, the network interface 160 includes aconventional network device (e.g., a router), and the smart homeenvironment 100 of FIG. 1 includes a hub device 180 that iscommunicatively coupled to the network(s) 162 directly or via thenetwork interface 160. The hub device 180 is further communicativelycoupled to one or more of the above intelligent, multi-sensing,network-connected devices (e.g., smart devices of the smart homeenvironment 100). Each of these smart devices optionally communicateswith the hub device 180 using one or more radio communication networksavailable at least in the smart home environment 100 (e.g., ZigBee,Z-Wave, Insteon, Bluetooth, Wi-Fi and other radio communicationnetworks). In some implementations, the hub device 180 and devicescoupled with/to the hub device can be controlled and/or interacted withvia an application running on a smart phone, household controller,laptop, tablet computer, game console or similar electronic device. Insome implementations, a user of such controller application can viewstatus of the hub device or coupled smart devices, configure the hubdevice to interoperate with smart devices newly introduced to the homenetwork, commission new smart devices, and adjust or view settings ofconnected smart devices, etc. In some implementations the hub deviceextends capabilities of low capability smart device to matchcapabilities of the highly capable smart devices of the same type,integrates functionality of multiple different device types—even acrossdifferent communication protocols, and is configured to streamlineadding of new devices and commissioning of the hub device. In someimplementations, hub device 180 further includes a local storage devicefor storing data related to, or output by, smart devices of smart homeenvironment 100. In some implementations, the data includes one or moreof: video data output by a camera device, metadata output by a smartdevice, settings information for a smart device, usage logs for a smartdevice, and the like.

In some implementations, smart home environment 100 includes a localstorage device 190 for storing data related to, or output by, smartdevices of smart home environment 100. In some implementations, the dataincludes one or more of: video data output by a camera device (e.g.,camera 118 or doorbell camera 106), metadata output by a smart device,settings information for a smart device, usage logs for a smart device,and the like. In some implementations, local storage device 190 iscommunicatively coupled to one or more smart devices via a smart homenetwork (e.g., smart home network 202, FIG. 2A). In someimplementations, local storage device 190 is selectively coupled to oneor more smart devices via a wired and/or wireless communication network.In some implementations, local storage device 190 is used to store videodata when external network conditions are poor. For example, localstorage device 190 is used when an encoding bitrate of camera 118exceeds the available bandwidth of the external network (e.g.,network(s) 162). In some implementations, local storage device 190temporarily stores video data from one or more cameras (e.g., camera118) prior to transferring the video data to a server system (e.g.,server system 164).

In accordance with various implementations of the application, adoorbell camera 106 integrates a camera 118 in a smart doorbell device106. The doorbell camera 106 has a doorbell button, a camera module, aprocessor and memory including programs executed by the processor, andis electrically coupled to a remote chime device that rings in responseto a user press on the doorbell button. The doorbell camera 106 operatesat a first camera mode and a second camera mode. In both of these twomodes, the doorbell camera 106 continuously records video informationfrom a field of view of the doorbell camera 106, and communicates with aremote server 164 to receive instructions from and/or upload therecorded video information to the remote server 164. More importantly,the doorbell camera 106 is configured to detect whether there is a userpress on the doorbell button. In accordance with a determination that nouser press is being applied on the doorbell button, the first cameramode is activated to bypass the remote chime device and couple thecamera module of the doorbell camera 106 to a remote transformer forreceiving a power supply therefrom. Conversely, in accordance with adetermination that a user press is being applied on the doorbell button,the second doorbell mode is activated to couple both the camera moduleof the doorbell camera 106 and the remote chime device to the remotetransformer. For example, the camera module and the remote chimer deviceare electrically coupled in series and both powered by the remotetransformer at the second doorbell mode, thereby enabling the remotechime device to ring concurrently while the camera module is recordingthe video information. In some implementations, while the doorbellbutton is being pressed (i.e., at a second doorbell mode), the cameramodule of the doorbell camera 106 is electrically decoupled from thetransformer, and relies on the battery to provide needed power. Thebattery is recharged at the first camera mode when the button is notpressed. The battery needs to be sized to be sufficiently large so thatit can charge back up in between button presses. In someimplementations, the doorbell button of the doorbell camera 106 isconfigured to sustain a predetermined number (e.g., 100) of continuouspresses without losing battery power.

In some implementations, the doorbell camera 106 is located at a door ofa structure 150, and the remote chime device and the transformer arelocated in two separate rooms 152 (e.g., in a kitchen 153 and a garage,respectively).

In some implementations, the doorbell camera 106 continues to operate asa standard doorbell (i.e., ring the remote chime device in response to auser press on the doorbell button), independently of whether the cameramodule of the doorbell camera 106 functions properly. For example, insome situations, the doorbell camera 106 loses connection to any localor wide area network. The camera module is disabled from capturingimages and sharing the captured images with the remote server 164 or anyclient device 166, and however, the doorbell camera 106 still respondsproperly to the user press on its doorbell button.

Additionally, in some implementations, video and audio recordingfunctions of the doorbell camera 106 are configured to comply with locallaws and regulations that are enforced in different jurisdictionsconcerning recording video and audio information in public placeswithout consent of those being recorded. The doorbell camera 106 ispre-programmed to comply with such laws and regulations in a factory,before it is shipped to a specific jurisdiction.

FIG. 2A is a block diagram illustrating a representative networkarchitecture 200 that includes a smart home network 202 in accordancewith some implementations. In some implementations, the smart devices204 in the smart home environment 100 (e.g., devices 102, 104, 106, 108,110, 112, 114, 116, 118, 120, and/or 122) combine with the hub device180 to create a mesh network in smart home network 202. In someimplementations, one or more smart devices 204 in the smart home network202 operate as a smart home controller. Additionally and/oralternatively, hub device 180 operates as the smart home controller. Insome implementations, a smart home controller has more computing powerthan other smart devices. In some implementations, a smart homecontroller processes inputs (e.g., from smart devices 204, electronicdevice 166, and/or server system 164) and sends commands (e.g., to smartdevices 204 in the smart home network 202) to control operation of thesmart home environment 100. In some implementations, some of the smartdevices 204 in the smart home network 202 (e.g., in the mesh network)are “spokesman” nodes (e.g., 204-1) and others are “low-powered” nodes(e.g., 204-9). Some of the smart devices in the smart home environment100 are battery powered, while others have a regular and reliable powersource, such as by connecting to wiring (e.g., to 120V or 240V linevoltage wires) behind the walls 154 of the smart home environment. Thesmart devices that have a regular and reliable power source are referredto as “spokesman” nodes. These nodes are typically equipped with thecapability of using a wireless protocol to facilitate bidirectionalcommunication with a variety of other devices in the smart homeenvironment 100, as well as with the server system 164. In someimplementations, one or more “spokesman” nodes operate as a smart homecontroller. On the other hand, the devices that are battery powered arethe “low-power-nodes. These nodes tend to be smaller than spokesmannodes and typically only communicate using wireless protocols thatrequire very little power, such as Zigbee, ZWave, 6LoWPAN, Thread,Bluetooth, etc.

In some implementations, some low-power nodes are incapable ofbidirectional communication. These low-power nodes send messages, butthey are unable to “listen.” Thus, other devices in the smart homeenvironment 100, such as the spokesman nodes, cannot send information tothese low-power nodes.

In some implementations, some low-power nodes are capable of only alimited bidirectional communication. For example, other devices are ableto communicate with the low-power nodes only during a certain timeperiod.

As described, in some implementations, the smart devices serve aslow-power and spokesman nodes to create a mesh network in the smart homeenvironment 100. In some implementations, individual low-power nodes inthe smart home environment regularly send out messages regarding whatthey are sensing, and the other low-powered nodes in the smart homeenvironment—in addition to sending out their own messages—forward themessages, thereby causing the messages to travel from node to node(i.e., device to device) throughout the smart home network 202. In someimplementations, the spokesman nodes in the smart home network 202,which are able to communicate using a relatively high-powercommunication protocol, such as IEEE 802.11, are able to switch to arelatively low-power communication protocol, such as IEEE 802.15.4, toreceive these messages, translate the messages to other communicationprotocols, and send the translated messages to other spokesman nodesand/or the server system 164 (using, e.g., the relatively high-powercommunication protocol). Thus, the low-powered nodes using low-powercommunication protocols are able to send and/or receive messages acrossthe entire smart home network 202, as well as over the Internet 162 tothe server system 164. In some implementations, the mesh network enablesthe server system 164 to regularly receive data from most or all of thesmart devices in the home, make inferences based on the data, facilitatestate synchronization across devices within and outside of the smarthome network 202, and send commands to one or more of the smart devicesto perform tasks in the smart home environment.

As described, the spokesman nodes and some of the low-powered nodes arecapable of “listening.” Accordingly, users, other devices, and/or theserver system 164 may communicate control commands to the low-powerednodes. For example, a user may use the electronic device 166 (e.g., asmart phone) to send commands over the Internet to the server system164, which then relays the commands to one or more spokesman nodes inthe smart home network 202. The spokesman nodes may use a low-powerprotocol to communicate the commands to the low-power nodes throughoutthe smart home network 202, as well as to other spokesman nodes that didnot receive the commands directly from the server system 164.

In some implementations, a smart nightlight 170 (FIG. 1), which is anexample of a smart device 204, is a low-power node. In addition tohousing a light source, the smart nightlight 170 houses an occupancysensor, such as an ultrasonic or passive IR sensor, and an ambient lightsensor, such as a photo resistor or a single-pixel sensor that measureslight in the room. In some implementations, the smart nightlight 170 isconfigured to activate the light source when its ambient light sensordetects that the room is dark and when its occupancy sensor detects thatsomeone is in the room. In other implementations, the smart nightlight170 is simply configured to activate the light source when its ambientlight sensor detects that the room is dark. Further, in someimplementations, the smart nightlight 170 includes a low-power wirelesscommunication chip (e.g., a ZigBee chip) that regularly sends outmessages regarding the occupancy of the room and the amount of light inthe room, including instantaneous messages coincident with the occupancysensor detecting the presence of a person in the room. As mentionedabove, these messages may be sent wirelessly (e.g., using the meshnetwork) from node to node (i.e., smart device to smart device) withinthe smart home network 202 as well as over the Internet 162 to theserver system 164.

Other examples of low-power nodes include battery-operated versions ofthe smart hazard detectors 104. These smart hazard detectors 104 areoften located in an area without access to constant and reliable powerand may include any number and type of sensors, such as smoke/fire/heatsensors (e.g., thermal radiation sensors), carbon monoxide/dioxidesensors, occupancy/motion sensors, ambient light sensors, ambienttemperature sensors, humidity sensors, and the like. Furthermore, smarthazard detectors 104 may send messages that correspond to each of therespective sensors to the other devices and/or the server system 164,such as by using the mesh network as described above.

Examples of spokesman nodes include smart doorbells 106, smartthermostats 102, smart wall switches 108, and smart wall plugs 110.These devices are often located near and connected to a reliable powersource, and therefore may include more power-consuming components, suchas one or more communication chips capable of bidirectionalcommunication in a variety of protocols.

In some implementations, the smart home environment 100 includes servicerobots 168 (FIG. 1) that are configured to carry out, in an autonomousmanner, any of a variety of household tasks.

As explained above with reference to FIG. 1, in some implementations,the smart home environment 100 of FIG. 1 includes a hub device 180 thatis communicatively coupled to the network(s) 162 directly or via thenetwork interface 160. The hub device 180 is further communicativelycoupled to one or more of the smart devices using a radio communicationnetwork that is available at least in the smart home environment 100.Communication protocols used by the radio communication network include,but are not limited to, ZigBee, Z-Wave, Insteon, EuOcean, Thread, OSIAN,Bluetooth Low Energy and the like. In some implementations, the hubdevice 180 not only converts the data received from each smart device tomeet the data format requirements of the network interface 160 or thenetwork(s) 162, but also converts information received from the networkinterface 160 or the network(s) 162 to meet the data format requirementsof the respective communication protocol associated with a targetedsmart device. In some implementations, in addition to data formatconversion, the hub device 180 further processes the data received fromthe smart devices or information received from the network interface 160or the network(s) 162 preliminary. For example, the hub device 180 canintegrate inputs from multiple sensors/connected devices (includingsensors/devices of the same and/or different types), perform higherlevel processing on those inputs—e.g., to assess the overall environmentand coordinate operation among the different sensors/devices—and/orprovide instructions to the different devices based on the collection ofinputs and programmed processing. It is also noted that in someimplementations, the network interface 160 and the hub device 180 areintegrated to one network device. Functionality described herein isrepresentative of particular implementations of smart devices, controlapplication(s) running on representative electronic device(s) (such as asmart phone), hub device(s) 180, and server(s) coupled to hub device(s)via the Internet or other Wide Area Network. All or a portion of thisfunctionality and associated operations can be performed by any elementsof the described system—for example, all or a portion of thefunctionality described herein as being performed by an implementationof the hub device can be performed, in different system implementations,in whole or in part on the server, one or more connected smart devicesand/or the control application, or different combinations thereof.

FIG. 2B illustrates a representative operating environment in which aserver system 164 provides data processing for monitoring andfacilitating review of events (e.g., motion, audio, security, etc.) invideo streams captured by video cameras 118 or doorbell cameras 106. Asshown in FIG. 2B, the server system 164 receives video data from videosources 222 (including cameras 118 or doorbell cameras 106) located atvarious physical locations (e.g., inside homes, restaurants, stores,streets, parking lots, and/or the smart home environments 100 of FIG.1). Each video source 222 may be bound to one or more reviewer accounts,and the server system 164 provides video monitoring data for the videosource 222 to client devices 220 associated with the reviewer accounts.For example, the portable electronic device 166 is an example of theclient device 220. In some implementations, the server system 164 is avideo processing server that provides video processing services to videosources and client devices 220.

In some implementations, each of the video sources 222 includes one ormore video cameras 118 or doorbell cameras 106 that capture video andsend the captured video to the server system 164 substantially inreal-time. In some implementations, each of the video sources 222includes a controller device (not shown) that serves as an intermediarybetween the one or more cameras and the server system 164. Thecontroller device receives the video data from the one or more cameras,optionally performs some preliminary processing on the video data, andsends the video data to the server system 164 on behalf of the one ormore cameras substantially in real-time. In some implementations, eachcamera has its own on-board processing capabilities to perform somepreliminary processing on the captured video data before sending theprocessed video data (along with metadata obtained through thepreliminary processing) to the controller device and/or the serversystem 164.

In accordance with some implementations, each of the client devices 220includes a client-side module. The client-side module communicates witha server-side module executed on the server system 164 through the oneor more networks 162. The client-side module provides client-sidefunctionality for the event monitoring and review processing andcommunications with the server-side module. The server-side moduleprovides server-side functionality for event monitoring and reviewprocessing for any number of client-side modules each residing on arespective client device 220. The server-side module also providesserver-side functionality for video processing and camera control forany number of the video sources 222, including any number of controldevices and the cameras.

In some implementations, the server system 164 includes one or moreprocessors 212, a video storage database 210, an account database 214,an I/O interface to one or more client devices 216, and an I/O interfaceto one or more video sources 218. The I/O interface to one or moreclients 216 facilitates the client-facing input and output processing.The account database 214 stores a plurality of profiles for revieweraccounts registered with the video processing server, where a respectiveuser profile includes account credentials for a respective revieweraccount, and one or more video sources linked to the respective revieweraccount. The I/O interface to one or more video sources 218 facilitatescommunications with one or more video sources 222 (e.g., groups of oneor more cameras and associated controller devices). The video storagedatabase 210 stores raw video data received from the video sources 222,as well as various types of metadata, such as motion events, eventcategories, event category models, event filters, and event masks, foruse in data processing for event monitoring and review for each revieweraccount.

Examples of a representative client device 220 include a handheldcomputer, a wearable computing device, a personal digital assistant(PDA), a tablet computer, a laptop computer, a desktop computer, acellular telephone, a smart phone, an enhanced general packet radioservice (EGPRS) mobile phone, a media player, a navigation device, agame console, a television, a remote control, a point-of-sale (POS)terminal, a vehicle-mounted computer, an ebook reader, or a combinationof any two or more of these data processing devices or other dataprocessing devices.

Examples of the one or more networks 162 include local area networks(LAN) and wide area networks (WAN) such as the Internet. The one or morenetworks 162 are implemented using any known network protocol, includingvarious wired or wireless protocols, such as Ethernet, Universal SerialBus (USB), FIREWIRE, Long Term Evolution (LTE), Global System for MobileCommunications (GSM), Enhanced Data. GSM Environment (EDGE), codedivision multiple access (CDMA), time division multiple access (TDMA),Bluetooth, Wi-Fi, voice over Internet Protocol (VoIP), Wi-MAX, or anyother suitable communication protocol.

In some implementations, the server system 164 is implemented on one ormore standalone data processing apparatuses or a distributed network ofcomputers. In some implementations, the server system 164 also employsvarious virtual devices and/or services of third party service providers(e.g., third-party cloud service providers) to provide the underlyingcomputing resources and/or infrastructure resources of the server system164. In some implementations, the server system 164 includes, but is notlimited to, a server computer, a handheld computer, a tablet computer, alaptop computer, a desktop computer, or a combination of any two or moreof these data processing devices or other data processing devices.

The server-client environment shown in FIG. 2B includes both aclient-side portion (e.g., the client-side module) and a server-sideportion (e.g., the server-side module). The division of functionalitybetween the client and server portions of operating environment can varyin different implementations. Similarly, the division of functionalitybetween a video source 222 and the server system 164 can vary indifferent implementations. For example, in some implementations, theclient-side module is a thin-client that provides only user-facing inputand output processing functions, and delegates all other data processingfunctionality to a backend server (e.g., the server system 164).Similarly, in some implementations, a respective one of the videosources 222 is a simple video capturing device that continuouslycaptures and streams video data to the server system 164 with limited orno local preliminary processing on the video data. Although many aspectsof the present technology are described from the perspective of theserver system 164, the corresponding actions performed by a clientdevice 220 and/or the video sources 222 would be apparent to one ofskill in the art. Similarly, some aspects of the present technology maybe described from the perspective of a client device or a video source,and the corresponding actions performed by the video server would beapparent to one of skill in the art. Furthermore, some aspects of thepresent technology may be performed by the server system 164, a clientdevice 220, and a video source 222 cooperatively.

In some implementations, a video source 222 (e.g., a camera 118 or 106)transmits one or more streams of video data to the server system 164. Insome implementations, the one or more streams may include multiplestreams, of respective resolutions and/or frame rates, of the raw videocaptured by the camera 118 or 106. In some implementations, the multiplestreams may include a “primary” stream with a certain resolution andframe rate, corresponding to the raw video captured by the camera 118 or106, and one or more additional streams. An additional stream may be thesame video stream as the “primary” stream but at a different resolutionand/or frame rate, or a stream that captures a portion of the “primary”stream (e.g., cropped to include a portion of the field of view orpixels of the primary stream) at the same or different resolution and/orframe rate as the “primary” stream.

In some implementations, one or more of the streams are sent from thevideo source 222 directly to a client device 220 (e.g., without beingrouted to, or processed by, the server system 164). In someimplementations, one or more of the streams is stored at the camera 118or 106 (e.g., in memory 406, FIG. 4) and/or a local storage device(e.g., a dedicated recording device), such as a digital video recorder(DVR). For example, in accordance with some implementations, the camera118 or 106 stores the most recent 24 hours of video footage recorded bythe camera. In some implementations, portions of the one or more streamsare stored at the camera 118 or 106 and/or the local storage device(e.g., portions corresponding to particular events or times ofinterest).

In some implementations, the server system 164 transmits one or morestreams of video data to a client device 220 to facilitate eventmonitoring by a user. In some implementations, the one or more streamsmay include multiple streams, of respective resolutions and/or framerates, of the same video feed. In some implementations, the multiplestreams include a “primary” stream with a certain resolution and framerate, corresponding to the video feed, and one or more additionalstreams. An additional stream may be the same video stream as the“primary” stream but at a different resolution and/or frame rate, or astream that shows a portion of the “primary” stream (e.g., cropped toinclude portion of the field of view or pixels of the primary stream) atthe same or different resolution and/or frame rate as the “primary”stream.

FIG. 3A is a block diagram illustrating the server system 164 inaccordance with some implementations. The server system 164 typicallyincludes one or more processing units (CPUs) 302, one or more networkinterfaces 304 (e.g., including an I/O interface to one or more clientdevices and an I/O interface to one or more electronic devices), memory306, and one or more communication buses 308 for interconnecting thesecomponents (sometimes called a chipset). The memory 306 includeshigh-speed random access memory, such as DRAM, SRAM, DDR SRAM, or otherrandom access solid state memory devices; and, optionally, includesnon-volatile memory, such as one or more magnetic disk storage devices,one or more optical disk storage devices, one or more flash memorydevices, or one or more other non-volatile solid state storage devices.The memory 306, optionally, includes one or more storage devicesremotely located from one or more processing units 302. The memory 306,or alternatively the non-volatile memory within memory 306, includes anon-transitory computer readable storage medium. In someimplementations, the memory 306, or the non-transitory computer readablestorage medium of the memory 306, stores the following programs,modules, and data structures, or a subset or superset thereof:

-   an operating system 310 including procedures for handling various    basic system services and for performing hardware dependent tasks;-   a network communication module 312 for connecting the server system    164 to other systems and devices (e.g., client devices, electronic    devices, and systems connected to one or more networks 162) via one    or more network interfaces 304 (wired or wireless);-   a server-side module 314, which provides server-side functionalities    for device control, data processing, and data review, including, but    not limited to:    -   a data receiving module 3140 for receiving data from electronic        devices (e.g., video data from a camera 118 or 106, FIG. 1) via        the hub device 180, and preparing the received data for further        processing and storage in the data storage database 3160;    -   a hub and device control module 3142 for generating and sending        server-initiated control commands to modify operation modes of        electronic devices (e.g., devices of a smart home environment        100), and/or receiving (e.g., from client devices 220) and        forwarding user-initiated control commands to modify operation        modes of the electronic devices;    -   a data processing module 3144 for processing the data provided        by the electronic devices, and/or preparing and sending        processed data to a device for review (e.g., client devices 220        for review by a user), including, but not limited to:        -   an event processor sub-module 3146 for processing event            candidates and/or events within a received video stream            (e.g., a video stream from cameras 118 or 106);        -   an event categorizer sub-module 3148 for categorizing event            candidates and/or events within the received video stream;            and        -   a user interface sub-module 3150 for communicating with a            user (e.g., sending alerts, timeline events, etc. and            receiving user edits and zone definitions and the like)-   a server database 316, including but not limited to:    -   a data storage database 3160 for storing data associated with        each electronic device (e.g., each camera) of each user account,        as well as data processing models, processed data results, and        other relevant metadata (e.g., names of data results, location        of electronic device, creation time, duration, settings of the        electronic device, etc.) associated with the data, where        (optionally) all or a portion of the data and/or processing        associated with the hub device 180 or smart devices are stored        securely;    -   an account database 3162 for storing account information for        user accounts, including user account information such as user        profiles 3163, information and settings for linked hub devices        and electronic devices (e.g., hub device identifications), hub        device specific secrets, relevant user and hardware        characteristics (e.g., service tier, device model, storage        capacity, processing capabilities, etc.), user interface        settings, data review preferences, etc., where the information        for associated electronic devices includes, but is not limited        to, one or more device identifiers (e.g., MAC address and UUID),        device specific secrets, and displayed titles;    -   a device information database 3164 for storing device        information related to one or more devices such as device        profiles 3165, e.g., device identifiers and hub device specific        secrets, independently of whether the corresponding hub devices        have been associated with any user account; and    -   an event information database 3166 for storing event information        such as event records 3168, e.g., event log information, event        categories, and the like.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, or modules, andthus various subsets of these modules may be combined or otherwiserearranged in various implementations. In some implementations, thememory 306, optionally, stores a subset of the modules and datastructures identified above. Furthermore, the memory 306, optionally,stores additional modules and data structures not described above.

FIG. 3B illustrates various data structures used by someimplementations, including an event record 3168-i, a user profile3163-i, and a device profile 3165-i. The event record 3168-i correspondsto an event i and data for the event i. In some instances, the data formotion event i includes event start data 31681 indicating when and/orhow the event started, event segments data 31682, raw video data 31683,event end data 31684 indicating when and/or how the event ended, eventfeatures data 31685, scene features data 31686, associated userinformation 31687, and associated devices information 31688. In someinstances, the event record 3168-i includes only a subset of the abovedata. In some instances, the event record 3168-i includes additionalevent data not shown such as data regarding event/motion masks.

The event start data 31681 includes date and time information such as atimestamp and optionally includes additional information such asinformation regarding the amount of motion present, a motion startlocation, amount of audio present, characteristics of the audio, and thelike. Similarly, the event end data 31684 includes date and timeinformation such as a timestamp and optionally includes additionalinformation such as information regarding the amount of motion present,a motion start location, amount of audio present, characteristics of theaudio, and the like.

The event segments 31682 includes information regarding segmentation ofmotion event i. In some instances, event segments are stored separatelyfrom the raw video data 31683. In some instances, the event segments arestored at a lower display resolution than the raw video data. Forexample, the event segments are optionally stored at 480p or 780p andthe raw video data is stored at 1080i or 1080p. Storing the eventsegments at a lower display resolution enables the system to devote lesstime and resources to retrieving and processing the event segments. Insome instances, the event segments are not stored separately and thesegmentation information includes references to the raw video data 31683as well as date and time information for reproducing the event segments.In some implementations, the event segments include one or more audiosegments (e.g., corresponding to video segments).

The event features data 31685 includes information regarding eventfeatures such as event categorizations/classifications, object masks,motion masks, identified/recognized/tracked motion objects (alsosometimes called blobs), information regarding features of the motionobjects (e.g., object color, object dimensions, velocity, size changes,etc.), information regarding activity in zones of interest, and thelike. The scene features data 31686 includes information regarding thescene in which the event took place such as depth map information,information regarding the location of windows, televisions, fans, theceiling/floor, etc., information regarding whether the scene is indoorsor outdoors, information regarding zones of interest, and the like. Insome implementations, the event features data includes audio data, suchas volume, pitch, characterizations, and the like.

The associated user information 31687 includes information regardingusers associated with the event such as users identified in the event,users receiving notification of the event, and the like. In someinstances, the associated user information 31687 includes a link,pointer, or reference to a user profile 3163 for to the user. Theassociated devices information 31688 includes information regarding thedevice or devices involved in the event (e.g., a camera 118 or 106 thatrecorded the event). In some instances, the associated devicesinformation 31688 includes a link, pointer, or reference to a deviceprofile 3165 for the device. In a specific example, the associated userinformation 31687 includes user identity of a visitor that has beenrecognized by the doorbell camera 106 when the visitor approaches thedoorbell camera 106 and knocks at the door.

The user profile 3163-i corresponds to a user i associated with thesmart home network (e.g., smart home network 202) such as a user of ahub device 204, a user identified by a hub device 204, a user whoreceives notifications from a hub device 204 or from the server system164, and the like. In some instances, the user profile 3163-i includesuser preferences 31631, user settings 31632, associated devicesinformation 31633, and associated events information 31634. In someinstances, the user profile 3163-i includes only a subset of the abovedata. In some instances, the user profile 3163-i includes additionaluser information not shown such as information regarding other usersassociated with the user i.

[The user preferences 31631 include explicit user preferences input bythe user as well as implicit and/or inferred user preferences determinedby the system (e.g., server system 164 and/or client device 220). Insome instances, the inferred user preferences are based on historicaluser activity and/or historical activity of other users. The usersettings 31632 include information regarding settings set by the user isuch as notification settings, device settings, and the like. In someinstances, the user settings 31632 include device settings for devicesassociated with the user i.

The associated devices information 31633 includes information regardingdevices associated with the user i such as devices within the user'ssmart home environment 100 and/or client devices 220. In some instances,associated devices information 31633 includes a link, pointer, orreference to a corresponding device profile 3165. Associated eventsinformation 31634 includes information regarding events associated withuser i such as events in which user i was identified, events for whichuser i was notified, events corresponding to user i's smart homeenvironment 100, and the like. In some instances, the associated eventsinformation 31634 includes a link, pointer, or reference to acorresponding event record 3168.

The device profile 3165-i corresponds to a device i associated with asmart home network (e.g., smart home network 202) such a hub device 204,a camera 118 or 106, a client device 220, and the like. In someinstances, the device profile 3165-i includes device settings 31651,associated devices information 31652, associated user information 31653,associated event information 31654, and environmental data 31655. Insome instances, the device profile 3165-i includes only a subset of theabove data. In some instances, the device profile 3165-i includesadditional device information not shown such as information regardingwhether the device is currently active.

The device settings 31651 include information regarding the currentsettings of device i such as positioning information, mode of operationinformation, and the like. In some instances, the device settings 31651are user-specific and are set by respective users of the device i. Theassociated devices information 31652 includes information regardingother devices associated with device i such as other devices linked todevice i and/or other devices in the same smart home network as devicei. In some instances, the associated devices information 31652 includesa link, pointer, or reference to a respective device profile 3165corresponding to the associated device.

The associated user information 31653 includes information regardingusers associated with the device such as users receiving notificationsfrom the device, users registered with the device, users associated withthe smart home network of the device, and the like. In some instances,the associated user information 31653 includes a link, pointer, orreference to a user profile 3163 corresponding to the associated user.

The associated event information 31654 includes information regardingevents associated with the device i such as historical events involvingthe device i. In some instances, the associated event information 31654includes a link, pointer, or reference to an event record 3168corresponding to the associated event.

The environmental data 31655 includes information regarding theenvironment of device i such as information regarding whether the deviceis outdoors or indoors, information regarding the light level of theenvironment, information regarding the amount of activity expected inthe environment (e.g., information regarding whether the device is in aprivate residence versus a busy commercial property), informationregarding environmental objects (e.g., depth mapping information for acamera), and the like.

FIG. 4 is a block diagram illustrating a representative smart device 204in accordance with some implementations. In some implementations, thesmart device 204 (e.g., any smart device of a smart home environment100, FIG. 1) includes one or more processing units (e.g., CPUs, ASICs,FPGAs, microprocessors, and the like) 402, one or more communicationinterfaces 404, memory 406, communications module 442 with radios 440,and one or more communication buses 408 for interconnecting thesecomponents (sometimes called a chipset). In some implementations, theuser interface 410 includes one or more output devices 412 that enablepresentation of media content, including one or more speakers and/or oneor more visual displays (e.g., a light ring formed on a periphery of afront cover plate, a button or a camera lens opening of a doorbellcamera). In some implementations, the user interface 410 also includesone or more input devices 414, including user interface components thatfacilitate user input such as a keyboard, a mouse, a voice-command inputunit or microphone, a touch screen display, a touch-sensitive input pad,a gesture capturing camera, a doorbell button or other input buttons orcontrols. Furthermore, some smart devices 204 use a microphone and voicerecognition or a camera and gesture recognition to supplement or replacethe keyboard. In some implementations, the smart device 204 includes oneor more image/video capture devices 418 (e.g., cameras, video cameras,scanners, photo sensor units).

The built-in sensors 490 include, for example, one or more thermalradiation sensors, ambient temperature sensors, humidity sensors, IRsensors, occupancy sensors (e.g., using RFID sensors), ambient lightsensors (e.g., the ambient light sensor (ALS) assembly 714 in FIG. 7A),motion detectors, accelerometers, and/or gyroscopes.

The radios 440 enable one or more radio communication networks in thesmart home environments, and allow a smart device 204 to communicatewith other devices. In some implementations, the radios 440 are capableof data communications using any of a variety of custom or standardwireless protocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread,Z-Wave, Bluetooth Smart, ISA100.5A, WirelessHART, MiWi, etc.) custom orstandard wired protocols (e.g., Ethernet, HomePlug, etc.), and/or anyother suitable communication protocol, including communication protocolsnot yet developed as of the filing date of this document.

The communication interfaces 404 include, for example, hardware capableof data communications using any of a variety of custom or standardwireless protocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread,Z-Wave, Bluetooth Smart, ISA100.5A, WirelessHART, MiWi, etc.) and/or anyof a variety of custom or standard wired protocols (e.g., Ethernet,HomePlug, etc.), or any other suitable communication protocol, includingcommunication protocols not yet developed as of the filing date of thisdocument.

The memory 406 includes high-speed random access memory, such as DRAM,SRAM, DDR RAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. The memory 406, or alternatively the non-volatilememory within the memory 406, includes a non-transitory computerreadable storage medium. In some implementations, the memory 406, or thenon-transitory computer readable storage medium of the memory 406,stores the following programs, modules, and data structures, or a subsetor superset thereof:

-   operating logic 420 including procedures for handling various basic    system services and for performing hardware dependent tasks;-   a device communication module 422 for connecting to and    communicating with other network devices (e.g., network interface    160, such as a router that provides Internet connectivity, networked    storage devices, network routing devices, server system 164, etc.)    connected to one or more networks 162 via one or more communication    interfaces 404 (wired or wireless);-   an input processing module 426 for detecting one or more user inputs    or interactions from the one or more input devices 414 and    interpreting the detected inputs or interactions;-   a user interface module 428 for providing and displaying a user    interface in which settings, captured data, and/or other data for    one or more devices (e.g., the smart device 204, and/or other    devices in smart home environment 100) can be configured and/or    viewed;-   one or more applications 430 for execution by the smart device    (e.g., games, social network applications, smart home applications,    and/or other web or non-web based applications) for controlling    devices (e.g., executing commands, sending commands, and/or    configuring settings of the smart device 204 and/or other    client/electronic devices), and for reviewing data captured by    devices (e.g., device status and settings, captured data, or other    information regarding the smart device 204 and/or other    client/electronic devices);-   a device-side module 432, which provides device-side functionalities    for device control, data processing and data review, including but    not limited to:    -   a command receiving module 4320 for receiving, forwarding,        and/or executing instructions and control commands (e.g., from a        client device 220, from a server system 164, from user inputs        detected on the user interface 410, etc.) for operating the        smart device 204;    -   a data processing module 4322 for processing data captured or        received by one or more inputs (e.g., input devices 414,        image/video capture devices 418, location detection device 416),        sensors (e.g., built-in sensors 490), interfaces (e.g.,        communication interfaces 404, radios 440), and/or other        components of the smart device 204, and for preparing and        sending processed data to a device for review (e.g., client        devices 220 for review by a user); and-   device data 434 storing data associated with devices (e.g., the    smart device 204), including, but is not limited to:    -   account data 4340 storing information related to user accounts        loaded on the smart device 204, wherein such information        includes cached login credentials, smart device identifiers        (e.g., MAC addresses and UUIDs), user interface settings,        display preferences, authentication tokens and tags, password        keys, etc.;    -   local data storage database 4342 for selectively storing raw or        processed data associated with the smart device 204 (e.g., video        surveillance footage captured by a camera 118 or 106);-   a bypass module 436 for detecting whether radio(s) 440 are    transmitting signals via respective antennas coupled to the radio(s)    440 and to accordingly couple radio(s) 440 to their respective    antennas either via a bypass line or an amplifier (e.g., a low noise    amplifier); and    -   a transmission access module 438 for granting or denying        transmission access to one or more radio(s) 440 (e.g., based on        detected control signals and transmission requests).

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, or modules, andthus various subsets of these modules may be combined or otherwiserearranged in various implementations. In some implementations, thememory 406, optionally, stores a subset of the modules and datastructures identified above. Furthermore, the memory 406, optionally,stores additional modules and data structures not described above.

In some implementations, the smart device 204 provides visual and/oraudio feedback to a person near the smart device 204. The feedback mayconcern an operational status of the smart device 204 itself, a statusof the outdoor environment surrounding the smart device 204, theoperational status of another electronic device associated with thesmart device 204, and/or the operational status of a set of electronicdevices associated with the smart device 204.

In some implementations, the smart device 204 includes a doorbell camera106, and the doorbell camera 106 has physical features that can providereal time camera status information and/or audio/visual content thatindicates or complements camera processing activity, to occupants of theenvironment without disturbing operation of the camera or the occupants.In some implementations, such physical features include a light ringthat is provided at a periphery of a front cover, a periphery of acamera lens, or a periphery of a button on the doorbell camera 106, andis configured to be visible to occupants of the environment from a widerange of positions in the environment. For example, in someimplementations, the light ring is configured to be visible in a rangeof positions that include at least areas of the environment that fallwithin the camera's field of view. In some implementations, the lightring has a plurality of individual lighting elements, each havingassociated lighting characteristics that are individually controllableto reflect local camera status and/or a camera processingstate/operation. In some configurations, the controllable lightingcharacteristics include one or more of on/off state, hue, saturationand/or brightness/intensity. In some configurations, the lightingelements are controlled individually to display an overall pattern(e.g., an entire ring or one or more portions of a ring) that can bestatic or dynamic (e.g., one or more rotating portions of a ring)consisting of a single displayed color or two or more differentdisplayed colors. Each of the patterns can conform to a visual languageand correspond to a camera status and/or a camera processing operation.For example, a color or a pattern of two or more different colors(static or dynamic) can indicate that the camera is on or off, has anactive or inactive connection to a server (e.g., a server that performsimage processing or that distributes video and notifications to remoteusers), is actively processing local information from the environment,or has received a notification or status information from another smartdevice in the home environment or a server. In some implementations thatinclude a speaker, the physical feature (e.g., a light ring) can becontrolled by the camera to display patterns that correspond to audiblebeats/rhythm of music or audio messages being played from the speaker ina range of colors selected to match the tempo/feeling of the music oraudio messages. Providing such information via light patterns isadvantageous as this is readily perceived by all/most users in theenvironment (even if they do not have access to camera smart phoneapplication) without intruding on activity of occupants in theenvironment, as audible alerts could do sometimes.

FIG. 5 illustrates a representative system architecture 500. In someimplementations, the server system 164 includes functional modules foran event processor 3146, an event categorizer 3148, and a user-facingfrontend 3150. The event processor 3146 obtains the event candidates(e.g., by processing the video stream, by receiving the event startinformation from the video source 222, or by detecting a user press on adoorbell button of a doorbell camera). In some implementations, theevent candidates include motion event candidates. In someimplementations, the event candidates include a user press on a doorbellbutton of a doorbell camera. In some implementations, the eventcandidates include audio and/or visual aspects. The event categorizer3148 categorizes the event candidates into different event categories.The user-facing frontend 3150 generates event alerts and facilitatesreview of the events by a reviewer through a review interface on aclient device 220. The client facing frontend also receives user editson the event categories, user preferences for alerts and event filters,and zone definitions for zones of interest. The event categorizeroptionally revises event categorization models and results based on theuser edits received by the user-facing frontend. The server system 164also includes a video and source data database 506, event categorizationmodels database 510, and event data and event masks database 511. Insome implementations, each of these databases is part of the serverdatabase 316 (e.g., part of data storage database 3160).

The server system 164 receives one or more video stream(s) 504 from thevideo source 222 and optionally receives event candidate information 502such as event start information (e.g., motion start information) andsource information 503 such as device settings for a camera 118 or 106(e.g., a device profile 3165 for camera 118 or 106). In someimplementations, the event processor sub-module 3146 communicates withthe video source 222. The server system sends alerts for events 512 andevent timeline information 513 to the client device 220. The serversystem 164 optionally receives user information from the client device220 such as edits on event categories 514 and zone definitions 515.

The data processing pipeline 516 processes video information (e.g., alive video feed) received from a video source 222 (e.g., including acamera 118 or 106 and an optional controller device) and/or audioinformation received from one or more smart devices in real-time toidentify and categorize events occurring in the smart home environment,and sends real-time event alerts and a refreshed event timeline to aclient device 220 associated with a reviewer account for the smart homeenvironment. The data processing pipeline 516 also processes storedinformation (such as stored video feeds from a video source 222) toreevaluate and/or re-categorize events as necessary, such as when newinformation is obtained regarding the event and/or when new informationis obtained regarding event categories (e.g., a new activity zone isobtained from the user).

After video and/or audio data is captured at a smart device (517), thedata is processed to determine if any potential event candidates arepresent. In some implementations, the data is initially processed at thesmart device (e.g., video source 222 or camera 118 or 106). Thus, insome implementations, the smart device sends event candidateinformation, such as event start information, to the server system 164.In some implementations, the data is processed at the server system 164for event start detection. In some implementations, the video and/oraudio data is stored on server system 164 (e.g., in video and sourcedata database 509). In some implementations, the video stream is storedon a server distinct from server system 164. In some implementations,after a motion start is detected, the relevant portion of the videostream is retrieved from storage (e.g., from video and source datadatabase 509).

In some implementations, the event identification process includessegmenting the video stream into multiple segments then categorizing theevent candidate within each segment. In some implementations,categorizing the event candidate includes an aggregation of backgroundfactors, entity detection and identification, motion vector generationfor each motion entity, entity features, and scene features to generatemotion features for the event candidate. In some implementations, theevent identification process further includes categorizing each segment,generating or updating an event log based on categorization of asegment, generating an alert for the event based on categorization of asegment, categorizing the complete event, updating the event log basedon the complete event, and generating an alert for the event based onthe complete event. In some implementations, a categorization is basedon a determination that the event occurred within a particular zone ofinterest. In some implementations, a categorization is based on adetermination that the event candidate involves one or more zones ofinterest. In some implementations, a categorization is based on audiodata and/or audio event characterization.

The event analysis and categorization process may be performed by thesmart device (e.g., the video source 222) and the server system 164cooperatively, and the division of the tasks may vary in differentimplementations, for different equipment capability configurations,and/or for different network and server load situations. After theserver system 164 categorizes the event candidate, the result of theevent detection and categorization may be sent to a reviewer associatedwith the smart home environment.

In some implementations, the server system 164 stores raw or compressedvideo data (e.g., in a video and source data database 509), eventcategorization models (e.g., in an event categorization model database510), and event masks and other event metadata (e.g., in an event dataand event mask database 511) for each of the video sources 222. In someimplementations, the video data is stored at one or more displayresolutions such as 480p, 780p, 1080i, 1080p, and the like.

In some implementations, the video source 222 (e.g., the camera 118 or106) transmits a live video feed to the remote server system 164 via oneor more networks (e.g., the network(s) 162). In some implementations,the transmission of the video data is continuous as the video data iscaptured by the camera 118 or 106. In some implementations, thetransmission of video data is irrespective of the content of the videodata, and the video data is uploaded from the video source 222 to theserver system 164 for storage irrespective of whether any motion eventhas been captured in the video data. In some implementations, the videodata may be stored at a local storage device of the video source 222 bydefault, and only video portions corresponding to motion eventcandidates detected in the video stream are uploaded to the serversystem 164 (e.g., in real-time).

In some implementations, the video source 222 dynamically determines atwhat display resolution the video stream is to be uploaded to the serversystem 164. In some implementations, the video source 222 dynamicallydetermines which parts of the video stream are to be uploaded to theserver system 164. For example, in some implementations, depending onthe current server load and network conditions, the video source 222optionally prioritizes the uploading of video portions corresponding tonewly detected motion event candidates ahead of other portions of thevideo stream that do not contain any motion event candidates; or thevideo source 222 uploads the video portions corresponding to newlydetected motion event candidates at higher display resolutions than theother portions of the video stream. This upload prioritization helps toensure that important motion events are detected and alerted to thereviewer in real-time, even when the network conditions and server loadare less than optimal. In some implementations, the video source 222implements two parallel upload connections, one for uploading thecontinuous video stream captured by the camera 118 or 106, and the otherfor uploading video portions corresponding to detected motion eventcandidates. At any given time, the video source 222 determines whetherthe uploading of the continuous video stream needs to be suspendedtemporarily to ensure that sufficient bandwidth is given to theuploading of the video segments corresponding to newly detected motionevent candidates.

In some implementations, the video stream uploaded for cloud storage isat a lower quality (e.g., lower resolution, lower frame rate, highercompression, etc.) than the video segments uploaded for motion eventprocessing.

As shown in FIG. 5A, the video source 222 includes a camera 118 or 106,and an optional controller device. In some implementations, the camera118 or 106 includes sufficient on-board processing power to perform allnecessary local video processing tasks (e.g., cuepoint detection formotion event candidates, video uploading prioritization, networkconnection management, etc.), and the camera 118 or 106 communicateswith the server system 164 directly, without any controller deviceacting as an intermediary. In some implementations, the camera 118 or106 captures the video data and sends the video data to the controllerdevice for the necessary local video processing tasks. The controllerdevice optionally performs the local processing tasks for multiplecameras. For example, there may be multiple cameras in one smart homeenvironment (e.g., the smart home environment 100, FIG. 1), and a singlecontroller device receives the video data from each camera and processesthe video data to detect motion event candidates in the video streamfrom each camera. The controller device is responsible for allocatingsufficient outgoing network bandwidth to transmitting video segmentscontaining motion event candidates from each camera to the server beforeusing the remaining bandwidth to transmit the video stream from eachcamera to the server system 164. In some implementations, the continuousvideo stream is sent and stored at one server facility while the videosegments containing motion event candidates are send to and processed ata different server facility.

In some implementations, the smart device sends additional sourceinformation 503 to the server system 164. This additional sourceinformation 503 may include information regarding a device state (e.g.,IR mode, AE mode, DTPZ settings, etc.) and/or information regarding theenvironment in which the device is located (e.g., indoors, outdoors,night-time, day-time, etc.). In some implementations, the sourceinformation 503 is used by the server system 164 to perform eventdetection and/or to categorize event candidates. In someimplementations, the additional source information 503 includes one ormore preliminary results from video processing performed by the camera118 or 106 (e.g., categorizations, object recognitions, motion masks,etc.).

In some implementations, the video portion after an event start incidentis detected is divided into multiple segments. In some implementations,the segmentation continues until event end information (sometimes alsocalled an “end-of-event signal”) is obtained. In some implementations,the segmentation occurs within the server system 164 (e.g., by the eventprocessor module 3146). In some implementations, the segmentationincludes generating overlapping segments. For example, a 10-secondsegment is generated every second, such that a new segment overlaps theprior segment by 9 seconds.

In some implementations, each of the multiple segments is of the same orsimilar duration (e.g., each segment has a 10-12 second duration). Insome implementations, the first segment has a shorter duration than thesubsequent segments. Keeping the first segment short allows for realtime initial categorization and alerts based on processing the firstsegment. The initial categorization may then be revised based onprocessing of subsequent segments. In some implementations, a newsegment is generated if the motion entity enters a new zone of interest.

In some implementations, after the event processor module obtains thevideo portion corresponding to an event candidate, the event processormodule 3146 obtains background factors and performs motion entitydetection identification, motion vector generation for each motionentity, and feature identification. Once the event processor module 3146completes these tasks, the event categorizer module 3148 aggregates allof the information and generates a categorization for the motion eventcandidate. In some implementations, false positive suppression isoptionally performed to reject some motion event candidates before themotion event candidates are submitted for event categorization. In someimplementations, determining whether a motion event candidate is a falsepositive includes determining whether the motion event candidateoccurred in a particular zone. In some implementations, determiningwhether a motion event candidate is a false positive includes analyzingan importance score for the motion event candidate. The importance scorefor a motion event candidate is optionally based on zones of interestinvolved with the motion event candidate, background features, motionvectors, scene features, entity features, motion features, motiontracks, and the like.

In some implementations, the video source 222 has sufficient processingcapabilities to perform, and does perform, the background estimation,motion entity identification, the motion vector generation, and/or thefeature identification.

FIG. 6 is a block diagram illustrating a representative client device220 associated with a user account in accordance with someimplementations. The client device 220, typically, includes one or moreprocessing units (CPUs) 602, one or more network interfaces 604, memory606, and one or more communication buses 608 for interconnecting thesecomponents (sometimes called a chipset). Optionally, the client devicealso includes a user interface 610 and one or more built-in sensors 690(e.g., accelerometer and gyroscope). The user interface 610 includes oneor more output devices 612 that enable presentation of media content,including one or more speakers and/or one or more visual displays. Theuser interface 610 also includes one or more input devices 614,including user interface components that facilitate user input such as akeyboard, a mouse, a voice-command input unit or microphone, a touchscreen display, a touch-sensitive input pad, a gesture capturing camera,or other input buttons or controls. Furthermore, some the client devicesuse a microphone and voice recognition or a camera and gesturerecognition to supplement or replace the keyboard. In someimplementations, the client device includes one or more cameras,scanners, or photo sensor units for capturing images (not shown).Optionally, the client device includes a location detection device 616,such as a GPS (global positioning satellite) or other geo-locationreceiver, for determining the location of the client device.

The memory 606 includes high-speed random access memory, such as DRAM,SRAM, DDR SRAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. The memory 606, optionally, includes one or morestorage devices remotely located from one or more processing units 602.The memory 606, or alternatively the non-volatile memory within thememory 606, includes a non-transitory computer readable storage medium.In some implementations, the memory 606, or the non-transitory computerreadable storage medium of the memory 606, stores the followingprograms, modules, and data structures, or a subset or superset thereof:

-   an operating system 618 including procedures for handling various    basic system services and for performing hardware dependent tasks;-   a network communication module 620 for connecting the client device    220 to other systems and devices (e.g., client devices, electronic    devices, and systems connected to one or more networks 162) via one    or more network interfaces 604 (wired or wireless);-   an input processing module 622 for detecting one or more user inputs    or interactions from one of the one or more input devices 614 and    interpreting the detected input or interaction;-   one or more applications 624 for execution by the client device    (e.g., games, social network applications, smart home applications,    and/or other web or non-web based applications) for controlling    devices (e.g., sending commands, configuring settings, etc. to hub    devices and/or other client or electronic devices) and for reviewing    data captured by the devices (e.g., device status and settings,    captured data, or other information regarding the hub device or    other connected devices);-   a user interface module 622 for providing and displaying a user    interface in which settings, captured data, and/or other data for    one or more devices (e.g., smart devices 204 in smart home    environment 100) can be configured and/or viewed;-   a client-side module 628, which provides client-side functionalities    for device control, data processing and data review, including but    not limited to:    -   a hub device and device control module 6280 for generating        control commands for modifying an operating mode of the hub        device or the electronic devices in accordance with user inputs;        and    -   a data review module 6282 for providing user interfaces for        reviewing data processed by the server system 164; and-   client data 630 storing data associated with the user account and    electronic devices, including, but not limited to:    -   account data 6300 storing information related to both user        accounts loaded on the client device and electronic devices        (e.g., of the video sources 222) associated with the user        accounts, wherein such information includes cached login        credentials, hub device identifiers (e.g., MAC addresses and        UUIDs), electronic device identifiers (e.g., MAC addresses and        UUIDs), user interface settings, display preferences,        authentication tokens and tags, password keys, etc.; and    -   a local data storage database 6302 for selectively storing raw        or processed data associated with electronic devices (e.g., of        the video sources 222, such as a camera 118 or 106).

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise rearranged in various implementations. In someimplementations, the memory 606, optionally, stores a subset of themodules and data structures identified above. Furthermore, the memory606, optionally, stores additional modules and data structures notdescribed above.

Example Functions of a Doorbell Camera System

In some implementations, the server-side module 314 of the server system164 determines that a user press is being applied on a button of adoorbell camera 106, and in accordance with the determination, sends apush notification to a user mobile application 624 executed on a clientdevice 220. Optionally, when such a notification is pushed into theclient device 220, the user mobile application 624 is automaticallyactivated to display a user interface to receive a user response.Alternatively, in some implementations, the server system 164 detectsmotion or recognizes a face of a person in a field of view of thedoorbell camera 106 based on video images recorded by the camera 106. Inaccordance with the detection or the recognition, the server system 164sends a push notification to the user mobile application 624 executed onthe client device 220 in association with the doorbell camera 106.Optionally, facial recognition is implemented on device (i.e., locallyon the doorbell camera 106) and in cloud (e.g., remotely in the serversystem 164). In some implementations, when face recognition isimplemented locally, it is simplified according to local computationalcapability available at the doorbell camera 106. As a result of thefacial recognition, the push notification is sent to the client device220 indicating whether a known person has been recognized or a visitoris an unrecognizable person (e.g., a stranger). In some implementations,face recognition is only conducted after a user press on the doorbellbutton is detected.

Alternatively, in some implementations, face recognition is conductedbefore a user press on the doorbell button, and triggers a process tomonitor and detect the user press on the doorbell button. Specifically,a motion stream is obtained from a camera (e.g., the doorbell camera106) of a smart home environment 100. The doorbell camera 106 has afield of view of an entryway of the smart home environment 100. Themotion stream is a video stream depicting movement or informationregarding amount of motion in a scene. The server 164 determines basedon an analysis of the motion stream that a visitor is approaching theentryway, and performs a facial recognition operation based on one ormore frames of the motion stream. The server 164 then determines basedon an outcome of the facial recognition operation whether the person isknown to the smart home environment. A time window is initiated inresponse to the determination that a visitor is approaching, and thetime window is optionally predefined to correspond to a reasonableamount of time for the visitor to complete approaching/reaching theentryway. During the time window, the server 164 obtains contextualinformation from one or more sensors of the smart home environment 100,and determines whether an action from the visitor (e.g., a doorbellpress, a door knock) is detected within the time window. In accordancewith an action being detected within the time window, a first type ofresponse is initiated. Examples of the first type of response includesending a first type of notification to a user of a client device. Inaccordance with no action being detected within the time window, asecond type of response is initiated. Examples of the second type ofresponse include sending a second type of notification to a user of aclient device.

In some implementations, images captured by the doorbell camera 106 aredistorted (e.g., show a fisheye effect), because the doorbell camera 106uses a wide-angle lens assembly to capture the images. The images areprocessed in the server system 164 before they are sent to the clientdevice 220 for display in the user mobile application 624. Specifically,the distortion of the images is at least partially compensated beforethe images are displayed on the client device 220.

In some implementations, when the client device 220 loads the usermobile application 624 associated with the doorbell camera 106, theapplication 624 initiates a live video and/or audio connection to thedoorbell camera, and allows video and audio information to stream withina predetermined duration of time (e.g., <1 second). Further, in someimplementations, the user mobile application 624 displays an event listof events detected form the video and audio information received fromthe doorbell camera 106. Each event of the event list of events isrepresented by a thumbnail on a user interface of the user mobileapplication 624. In some implementations, the user mobile application624 stores location information of a plurality of cameras used in thesmart home environment 100, including the doorbell camera 106.

In some implementations, the doorbell camera 106, the client device 220and the remote server 164 allow a user to interact with a visitor to asmart home environment 100 via an electronic greeting system of thesmart home environment 100. The server 164 obtains motion data from asmart device (e.g., the doorbell camera 106), and identifies based onanalysis of the motion data a motion event involving a visitorapproaching an entryway (e.g., a door) of the smart home environment100. The server 164 obtains context information from the smart homeenvironment 100 for the motion event, and based on the contextinformation, identifies a plurality of appropriate actions available toa user of a client device for interacting with the visitor via thedoorbell camera 106. The identified actions are presented to the user ofthe client device 220. For example, the appropriate actions include, butare not limited to, displaying a visual pattern on a light ring of thedoorbell camera 106, broadcasting an audio message to the visitor, andcollecting more information concerning the visitor using the doorbellcamera 106.

The doorbell integrated in the doorbell camera 106 can also takeadvantage of computational and communication capabilities of thedoorbell camera 106. In some implementations, a schedule can be definedon the user mobile application 624 of the client device 220 to control aremote chime device that rings in response to a user press on thedoorbell button of the doorbell camera 106. For example, in accordancewith the schedule, the remote chime device is controlled to remainsilent from 12 PM to 4 PM every day, thereby allowing a homeowner tohave a quiet afternoon nap.

Doorbell Camera

FIGS. 7A and 7B are a front view and a rear view of a doorbell camera106 in accordance with some implementations. The doorbell camera 106integrates a camera module and a doorbell system within a device housing702. The camera module includes a lens assembly 704 and an image sensorarray that are configured to capture images, and a wireless transceiverthat is configured to exchange data with a remote server over one ormore wireless communication networks. The doorbell system is configuredto ring a remote chimer device in response to a press on a button top706 of a button assembly. The device housing 702 has a substantiallyelongated shape. Optionally, the elongated shape has a width that isless than a width of most doorframes available in the market, such thatthe doorbell camera 106 can be directly mounted on a commonly availabledoorframe. In some implementations, the lens assembly 704 of the cameramodule is disposed within a top half of the device housing 702, and thebutton top 706 of the doorbell system is disposed on a bottom half ofthe device housing 702. When the doorbell camera 106 is mounted on awall surface or a doorbell frame, the lens assembly 704 is located ontop of the button top 706, thereby allowing a field of view of the lensassembly 704 to remain clear when a user presses the button top 706.

Referring to FIGS. 7A and 7B, in this example, the device housing 702includes an elongated body and two semicircular ends, and the elongatedbody has a width substantially equal to a diameter of the twosemicircular ends. The lens assembly 704 is concentric with a top one ofthe two semicircular ends, and the button top 706 is concentric with abottom one of the two semicircular ends. Such physical arrangements notonly create a more aesthetically pleasing look for the doorbell camera106, but also ensure sufficient separation between the lens assembly 702and the button top 706, thereby avoiding a user press on the button top706 to interfere with video recording of the camera module (e.g., blockthe field of view of the lens assembly 704).

In some implementations, the device housing 702 includes a front coverplate 708 that seals at least a plurality of electronic components, thelens assembly and a button assembly within the device housing 702.Optionally, the front cover plate 708 has a peripheral edge thatinterfaces with a body of the device housing 702. The peripheral edge isconfigured to form a water tight seal with an edge of the body of thedevice housing 702, thereby deterring water from entering the devicehousing 702 through the interface between the front cover plate 708 andthe body of the device housing 702.

The button top 706 is part of the button assembly, and is configured toreceive the user press on the button assembly. The front cover plate 708therefore includes an button opening 710 configured to expose the buttontop 706 of the button assembly on the front cover plate 708. The buttontop 706 has a first diameter that is substantially smaller than a seconddiameter of the opening 710 of the front cover plate 708. The button top706 can move vertically in a direction perpendicular to a plane of thebutton opening 710 (e.g., in a downward direction into or out of thedevice housing 702) when a user presses on the button top 706. Thebutton assembly includes a flexible edge that surrounds the button top706 and fills a gap between the button opening 710 and the button top706. The flexible edge enables a range of motion for movement of thebutton top 706 with respect to the button opening 710 of the front coverplate 708. In some implementations, the flexible edge of the buttonassembly forms a water tight seal with an edge of the button opening 710of the front cover plate 708, and deters water from entering the devicehousing 702 via the opening 710.

In some implementations, the front cover plate 708 of the device housing702 is made of a single piece of material (e.g., glass and plastic). Thesingle piece of material creates a more aesthetically pleasing look,reduces production costs by reducing the number of parts, reducescomplexity by eliminating the need to tightly fit multiple sections,increases waterproofing of the device by eliminating seams betweenmultiple sections, and increases a quality of the images captured byimage sensors.

In some implementations, the front cover plate 708 includes a cameraopening 712 configured to expose the lens assembly 704. The lensassembly 704 includes a cover glass 705 configured to protect the lensassembly 704. An edge of the cover glass 705 of the lens assembly 704forms a watertight seal with an edge of the camera opening 712.Alternatively, in some implementations, the front cover plate 708 doesnot have the camera opening 712, but includes a substantiallytransparent area 712 that is disposed on top of the lens assembly 704 toallow light (e.g., visible and infrared light) to enter the lensassembly 704. In some implementations, the camera opening or transparentarea 712 has a size substantially larger than that of lenses of the lensassembly 704. By these means, the camera opening or transparent area 712of the front cover plate 708 does not become a limiting factor for amargin of a field of view of the doorbell camera 106, and the size ofthe lens assembly 704 determines the field of view for the doorbellcamera 106. Stated another way, the margin of the field of view islimited by physical features of the lens assembly 704, rather than bythe camera opening or transparent area 712 of the front cover plate 708.Such a substantially large camera opening or transparent area 712enables a substantially wide angle view for the doorbell camera 106 whena wide angle lens assembly is applied.

In some implementations, the front cover plate 708 includes asubstantially opaque area that is distinct from the substantiallytransparent area of the camera opening 712. An interior surface of thesubstantially opaque area is painted with dark color ink (e.g., blackink). The dark color ink could be transparent to infrared light, suchthat the substantially opaque area is opaque to visible light andtransparent to infrared light. In an example, the interior surface ofthe front cover plate 708 is entirely covered by the ink except thebutton opening 710 exposing the button top 706 and the substantiallytransparent area 712 covering the lens assembly 704. Further, in someimplementations, the doorbell camera 106 includes an array of IRilluminators (not shown in FIGS. 7A and 7B). The IR illuminators can bedisposed below any portion behind the substantially opaque area. The IRilluminators are concealed under the front cover plate 708, but lightgenerated by the IR illuminators can still penetrate the front coverplate 708 to illuminate the field of view of the doorbell camera 106.

In some implementations, the doorbell camera 106 further includes anambient light sensor (ALS) assembly 714 at least partially disposedwithin the lens assembly 704. The ALS assembly 714 is configured tomeasure amount of light entering the lens assembly 702 from an ambientenvironment surrounding the lens assembly 704. Part of the ALS assembly714 is exposed to incoming light, and therefore, visible from a front ofthe doorbell camera 106 in the lens assembly 704. The ALS assembly 714is configured to compare the measured amount of light to a thresholdillumination level, and the doorbell camera uses correspondingcomparison results to determine its operation mode between a daytimemode and a night mode. Specifically, in some situations, the ALSassembly 714 determines that the mount of light entering the lensassembly 702 exceeds the threshold illumination level, and the doorbellcamera 106 enables the daytime mode in which the IR illuminators areturned off. Alternatively, in some situations, the ALS assembly 714determines that the amount of light entering the lens assembly 702 isnot greater than the threshold illumination level, and the doorbellcamera 106 enables a night mode in which the IR illuminators areoptionally turned on to illuminate the field of view of the doorbellcamera 106. In some implementations, two distinct threshold illuminationlevels are applied to enable a first type of switching from the daytimemode to the night mode and a second type of switching from the nightnode to the daytime mode. Thus, use of the ALS assembly 714 can be usedto cause switching between the daytime and night modes andenable/disable the IR illuminators in an efficient manner.

The doorbell camera 106 further includes a microphone and a speaker. Insome implementations, the front cover plate 708 includes a microphoneaperture 716 to allow sound signals to reach the microphone concealedwithin the doorbell camera 106. In some implementations, the devicehousing 702 includes a plurality of speaker opening 718 at its bottomrim surface. When the doorbell camera 106 is mounted onto a wall ordoorframe surface, the speaker holes 718 are not visible from a gaze ofa visitor approaching or standing near the surface, while still beingable to broadcast audio messages to the visitor. Given that the doorbellcamera 106 includes both the microphone and the speaker, a remote usermay review live video streams captured by the camera module of thedoorbell camera 106, and have a conversation in real-time with thevisitor.

In some implementations, both the microphone and the speaker of thedoorbell camera 106 adopt waterproof features to deter water permeationinto the electronic components within the device housing 702 and causeirreversible damages to the electronic components. Specifically, thewaterproof features are integrated at the openings 716 and 718 to deterwater permeation (such as from a jet or stream of water impinging on atleast one of the openings 716 and 718). In an example, the microphone isdisposed inside the device housing 102, and has a sound input regionoffset from the microphone aperture 716. A hydrophobic membrane isaffixed to a first interior surface of the device housing 702 and coversthe microphone aperture 716 thereon. The hydrophobic membrane isconfigured to allow transmission of sound waves and block waterintrusion from the microphone aperture 716. A sound transmission channelcouples the sound input region of the microphone to the microphoneaperture 716 on the front cover plate 708. The sound transmissionchannel is configured to allow sound waves transmitted through themicrophone aperture 716 and the hydrophobic membrane to be coupled tothe sound input region of the microphone without exposing the soundinput region to damaging pressures due to environmental impacts on thedoorbell camera 106. In some implementations, the speaker may also adopta sound input region offset from the speaker openings 718 and arespective hydrophobic membrane to block water intrusion while allowingtransmission of sound waves from the speaker openings 718 to the soundinput region of the speaker. More details of some examples of waterproofmicrophones and speakers are discussed in U.S. patent application Ser.No. 15/209,735, filed Jul. 13, 2016, titled “Magnetic Mount Assembly ofa Camera,” which is hereby incorporated by its entirety.

Referring to FIG. 7B, in some implementations, a rear exterior surfaceof the device housing 702 provides an access to at least one of aplurality of wire terminals 720 and an electrical connector 722 (e.g., aUniversal Serial Bus (USB) connector). The plurality of wire terminals720 include a first terminal and a second terminal configured to receivea power supply line and a ground line for driving the plurality ofelectronic components contained within the device housing 106.Specifically, in an example, each of the power supply line and theground line includes a respective fork terminal coupled to acorresponding wire terminal 720, and fixed thereto by tightening a screwfastener. Under some circumstances, the doorbell camera 106 is mountedon a wall or doorframe surface in place of a conventional doorbell, andthe plurality of wire terminals 722 are coupled to receive the powersupply line and the ground line that were applied to drive theconventional doorbell.

Alternatively, in some implementations, the electrical connector 722 iscoupled to one or more electrical wires for receiving a power supply orexchanging data with another electronic device. In an example, both theplurality of wire terminals 720 and the electrical connector 722 areavailable on the rear exterior surface of the device housing 702. Theelectrical connector 722 is only connected in a factory for testing,calibrating and/or setting up the doorbell camera 106, while theplurality of wire terminals 720 are applied to provide the power supplyin the factory and/or after the doorbell camera 106 is shipped to acustomer.

In some implementations, the rear exterior surface of the device housing702 includes a plurality of recesses 724. For example, the plurality ofwire terminals 720 is disposed within a first recess 724A, and separatedby a separation 726 that protects the lines connected to the first andsecond terminals of the plurality of wire terminals 720 from beingshorted to each other. A height of the separation 726 is smaller than adepth of the first recess, and the depth of the first recess 724A isconfigured to be greater than a height of a connection formed betweenthe plurality of wire terminals 720 and the power supply or ground line.Specifically, in the above example, when the screw fastener is used tofix the fork terminal of the power supply or ground line onto one of theplurality of wire terminals 720, the depth of the first recess 724A isconfigured to be greater than a total height summing a thickness of thefork terminal and a height of a screw head of the screw fastener. Assuch, when the power supply and ground lines are connected (i.e.,tightened) to the plurality of wire terminals 720, the lines extendbeyond the first recess 724A, while the corresponding connections (i.e.,at the plurality of wire terminals 720) do not rise beyond the rearexterior surface of the device housing 702.

Doorbell Camera With Battery at Chime

FIG. 8 is a schematic block diagram of a doorbell camera system 800 inaccordance with some implementations. Doorbell camera system 800 caninclude doorbell camera subsystem 801, transformer 810, high voltagepower source 820, and chime subsystem 830. Doorbell camera subsystem 801may be placed on an exterior surface of a structure, and transformer810, high voltage power source 820, and chime system 830 may be placedinside of the structure. Doorbell camera subsystem 801 may includedoorbell button 802, camera module 804, an LED indicator (not shown), aspeaker (not shown), a microphone (not shown), a processor and memory(not shown) including programs executed by the processor. Camera module804 can include a lens assembly and an image sensor array that areconfigured to capture images at a premises, and a wireless transceiverthat is configured to exchange data with a remote server over one ormore wireless communication networks. Doorbell button 802 can beconfigured to trigger a remote chime 836 in response to a user press ofbutton 802. In some implementations, the LED indicator is configured toilluminate through a peripheral edge of the doorbell button 802.

High voltage power source 810 may be an AC line power source (e.g., 110v AC) received from a circuit breaker box within the structure.Transformer 820 may be coupled to power source 810, doorbell camerasubsystem 801, and chime subsystem 830. Transformer 820 may transformone or more characteristics of the power signal received from powersource 810 to have one or more different characteristics (voltage and/orcurrent) of the power signal that exits transformer 820. For example,transformer 820 may step down voltage of the power signal received onits primary windings to a lower voltage power signal that is output onits secondary windings such that camera doorbell 801 and chime system830 are supplied with the lower voltage power signal. In system 800,power provided by transformer 820 is operative to power doorbell camera803 and chime subsystem 830.

Chime subsystem 830 may include electronics 832, battery 834, and chime836. Electronics 832 may include any electronics such as a processor,power regulating circuitry, and other circuitry (that is described inmore detail below). Battery 834 may be a rechargeable battery. Chime 836may be any suitable apparatus capable of producing a doorbell sound inresponse to a button press on button 802. For example, chime 836 may bea speaker. As another example, chime 836 may be an electrical mechanicaldevice that rings one or more bells or chimes.

Doorbell camera subsystem 801 may communicate with chime system 830 viapower line communications using power lines 840 and 841 or via bypasscommunications line 842. In one embodiment, bypass communications line842 may be a separate wire coupling doorbell camera 802 to chimesubsystem 830 is electrically coupled to chime subsystem 830. In anotherembodiment, bypass communications line 842 may splice or tap into anexisting connection between doorbell camera 801 and transformer 820,between transformer 820 and chime subsystem 830, or between doorbellcamera subsystem 801 and chime subsystem 830.

In another approach, bypass unit 850 may be used in system 800. Ifbypass unit is used, power line connection 840 is severed, andtransformer 820 may be connected to bypass unit 850 via line 851, bypassunit 850 may be connected to doorbell camera subsystem 801 via line 852,and bypass unit 850 may be connected to chime subsystem 830 via lines853 and 854.

It should be appreciated that although the power line connections inFIG. 8 are represented by single lines, multiple lines may exist. Forexample, two lines may exist to provide a return path. As anotherexample, bypass line 842 may be used as another power line to coupledoorbell camera 801 to chime system 830. Such a power line may form acurrent loop that starts with transformer 820, proceeds to doorbellcamera 801, and then to chime system 830, and returns to transformer820.

In yet another approach, doorbell camera subsystem 801 may communicatewith chime subsystem 830 via wireless communications. The wirelesscommunications can include several radios each of which configured forone of broadband (e.g., Wi-Fi, cellular, etc.) communications,point-to-point (e.g., Bluetooth) communications, and mesh networking(e.g., Thread, Zigbee, ZWave, IEEE 802.15.4, etc.) communications.

Contrary to legacy doorbell camera systems that co-locate a battery withthe camera, the doorbell camera system according to embodimentsdescribed herein co-locates the battery with the chime subsystem and notwith the camera subsystem. In this approach, camera 804 is alwayspowered by line power and chime 836 is activated by a battery that isrecharged using line power in between button presses. Moving the batteryaway from doorbell camera subsystem can enable enhanced designflexibility for the doorbell camera subsystem. For example, theindustrial design doorbell camera body can be shrunk (i.e., because nobattery is present), and additional features (e.g., improved camerasensor, less feature throttling, improved speaker quality, and wirelesscommunications with other devices) can be added because the doorbellcamera system is no longer reliant on a battery. Moreover, by moving thebattery away from the doorbell camera subsystem, the battery itself isno longer exposed to possible high temperature fluctuations outside. Inaddition, the battery can be sized bigger because it is no longerconfined to the size limitations of a doorbell housing. The battery canbe positioned within the structure, for example, such as near thetransformer. Furthermore, because the battery is co-located with thechime, additional features relating to the chime may be employed thatwere not previously possible with legacy doorbell camera systems. Forexample, the legacy chime (e.g., house chime) may be replaced orsupplemented with a new chime such as a speaker.

Doorbell camera system 800 can operate according to different modes ofoperations. In a first mode of operation, chime 836 is only powered bybattery 834. Thus, during a button press event, power supplied by powersource 810 may supply power to battery 834 and battery 834 may supplypower to chime 836. During non-button press events, power supplied bypower source 810 may supply power to battery 834. In a second mode ofoperation, chime 836 is simultaneously powered by power source 810 andbattery 834. Thus, during a doorbell button press event, power source810 and battery 834 are used to power chime 836, and during non-buttonpress events, power source 810 is used to power battery 834. Electronics832 may control usage of power as appropriate for both modes ofoperation.

FIG. 9 shows an illustrative current diagram of a doorbell system 900according to an embodiment. As shown, doorbell system 900 may includeseveral bubbles that represent current draw or supply of differentcomponents of system 900. In particular, system 900 includes powersource current bubble 910, camera current bubble 920, chime currentbubble 930, all of which are in series with each other, and compensationnetwork current bubble 940, which is in parallel with bubble 930. Powersource current bubble 910 may be representative of current supplied byto transformer 820 (of FIG. 8), camera current bubble 920 may berepresentative of current consumed by doorbell camera subsystem 801 (ofFIG. 8), and chime current bubble 930 may be representative of currentconsumed by a portion of chime subsystem 830 (of FIG. 8). Compensationnetwork current bubble 940 may representative of current consumed by acompensation network that may be integrated with chime subsystem 830.According to Kirchhoff s law, the sum of the current flowing into aparticular node is equal to the sum of currents flowing out of thatnode. Thus, the current (I) flowing out of power source current bubble910 should be the same as the current flowing into power source currentbubble 910. This current (I) flows through camera current bubble 920 andthe parallel combination of chime current bubble 930 and compensationnetwork current bubble 940. The current consumed by the camera(I_(camera)) may be approximately the same as (I). The current consumedby chime (I_(chime)) may not be approximately the same as (I). In someembodiments, (I_(chime)) may be less than (I). A current compensationnetwork, represented by bubble 940, may balance the current drawn by(I_(chime)) by consuming the appropriate quantity of current(I_(compensation)) required to ensure that that combined currentconsumption of bubbles 930 and 940 is approximately the same as (I).Thus, by compensating for the lesser current draw of (I_(chime)) via thecurrent compensation network, Kirchoff s law is adhered to and thecurrent (I) is returned to power source current bubble 910. The currentcompensation network can serve as a current balancing circuit to ensurethat the current drawn by (I_(camera)) and is balanced with the currentdrawn by the sum of (I_(chime)) and (I_(compensation)). Balancing thecurrent draw between (I_(camera)) and (I_(chime)) can preventmalfunctions in doorbell system 900.

The compensation network also ensures that voltages are balance betweenthe camera and chime so that they both stay within their respectiveoperating voltage ranges. The voltages for power source (V_(SRC)),camera (V_(CAM)), compensation network (V_(COMP)), and chime (V_(chime))are shown. V_(SRC) is equal to the sum of V_(CAM) and V_(COMP), andV_(COMP) is equal to V_(CHIME). The compensation circuitry can adjustits current compensation, which in turn adjusts V_(COMP) to maintainV_(CAM) and V_(CHIME) in the appropriate their respective voltageranges.

FIG. 10 shows an illustrative schematic diagram of doorbell system 1000in accordance with an embodiment. System 1000 can include power source1010, camera subsystem 1020, and chime subsystem 1050. Camera subsystem1020 may be operative to provided doorbell button press detection andcamera functionality. Chime subsystem 1050 may be operative to operate adoorbell chime. Current may flow from power source 1010 to camerasubsystem 1020, and then to chime subsystem 1050, and back to powersource 1010. Power source 1010 may be a transformer that is coupled toutility power supply line. The power provide by power source 1010 may beprovided in alternating current.

Camera subsystem 1020 can include power conditioning circuitry 1024,processor 1028, doorbell 1030, camera 1034, and doorbell button presstrigger 1036. Power conditioning circuitry 1024 can be operative tocondition the AC power signal received from power source 1010 to a DCpower signal suitable for the operation of processor 1028, doorbell1030, and camera 1034. In some embodiments, power conditioning circuitry1024 can include rectifier 1025 and buck converter 1026. Rectifier 1025may be a full-bridge rectifier, for example. Rectifier 1025 may convertan AC power signal to a DC power signal. Buck converter 1026 may beoperative to step down and regulate the voltage level of the DC powersignal provided by rectifier 1025. Buck converter 1026 may use powerelectronics that use a power switching transistor to modulate the DCpower signal to a desired voltage level. It should be appreciated thatmany different power conditioning circuit arrangements may be used tocondition the power signal. For example, a boost converter, buck-boostconverter, or a boost-buck converter may be used in place of buckconverter 1026.

The DC regulated power supplied by power conditioning circuitry 1024 isprovided to processor 1028 and camera 1034. This conditioned power canensure that processor 1028 and camera 1034 are provided with the powerrequired to perform their respective operations. When camera 1034 isactive, power conditioning circuitry 1024 is operative to supplyuninterrupted power to camera 1034 so that it can operate as intended,regardless of environmental conditions impacting camera subsystem 1020.

The arrangement of camera subsystem 1020 is different than otherconfigurations in that there is no battery associated with camerasubsystem 1020. The battery has been moved to chime subsystem 1050(discussed below). Locating the battery away from camera subsystem 1020eliminates thermal cycling issues that may plague the battery due to itsproximity external to the structure. For example, if camera subsystem1020 is exposed to sun light, it may be exposed to relatively hightemperatures that affect the battery.

Chime subsystem 1050 can include current compensation network 1052,power conditioning circuitry 1054, processor 1058, battery charger/boostconverter 1060, battery 1062, chime driver circuitry 1064, chime 1066,and doorbell button press sensor 1068. Current compensation network 1052may be operative to balance the current draw of chime subsystem 1050with the current drawn by camera subsystem 1020. Current compensationnetwork 1052 can adjust a quantity of current consumed by chimesubsystem 1050 by dynamically varying the amount of current it consumesbased on operation of other components within chime subsystem 1050 orthe current being consumed by camera subsystem 1020. Different examplesof current compensation networks are discussed below in more detail.

In one embodiment, doorbell button press trigger 1036 and doorbellbutton press sensor 1068 may be coupled to the power line to performpower line communications among camera subsystem 1020 and chimesubsystem 1050. In other embodiments, trigger 1036 and sensor 1068 maycommunicate with each other through other methods such as by a wirelessradio frequency signal.

Currently compensation network 1052 is shown being connected to a returnpath of rectifier 1025 and to power conditioning circuitry 1054, and inparticular to rectifier 1055 (of circuitry 1054). Power conditioningcircuitry 1054 may include rectifier 1055 (e.g., a full bridgerectifier) and buck converter 1056. Power conditioning circuitry 1054may convert an AC power signal (received from power source 1010) to a DCpower signal suitable for processor 1058. The DC power signal may beprovided to battery charger/boost converter 1060, which may supply powerwith an appropriate voltage to charge battery 1062 and may also supplypower with the appropriate voltage to enable chime driver circuitry 1064to activate chime 1066. Processor 1058 may be coupled to chime drivercircuitry 1064 and operative to cause driver circuitry 1064 to activatechime 1066. For example, when the button at doorbell 1030 is pressed,processor 1028 may detect the button press event and cause doorbellbutton press trigger 1036 to communicate the doorbell event over thepower line connected to chime subsystem 1050. Doorbell button presssensor 1068 may detect the doorbell event on the power line andcommunicate the detected event to processor 1058, which then causeschime 1066 to activate by communicating with driver circuitry 1064.

When chime 1066 is activated, it may require more current to operatethan that available from power source 1010. Thus, when chime activationis requested, battery 1062 may supply the supplemental current requiredby chime 1066. Since battery 1062 is located with chime subsystem 1050,which is typically located within the structure, it is not subject totemperature fluctuations that it would otherwise be subject to if itwere included as part of camera subsystem 1020.

It should be understood that in some embodiments, components of chimesubsystem 1050 may arranged in a different configuration. For example,processor 1058 may be coupled to receive power from charger boost 1060as opposed to buck converter 1056.

FIG. 11 shows an illustrative timing diagram showing current consumptionof various components within doorbell system 1000 according to anembodiment. Starting at time, to, doorbell system 1000 may operate instandby or NO doorbell mode and a doorbell mode. When operating in thestandby or NO doorbell event mode, camera subsystem 1020 may drawcurrent (I_(camera)) and chime subsystem 1050 may draw the sum ofcurrents (I_(compensation)) and (I_(chime)), where (I_(compensation))refers to current consumed by current compensation network 1052 and(I_(chime)) refers to current consumed by power conditioning circuitry1054, processor 1058, boost converter 1060, battery 1062, driver 1064,and chime 1066. Because chime 1066 is not active, the (I_(chime))current may be less than (I_(camera)). Current compensation network 1052can dynamically adjust how much current it consumes to balance thecurrent consumed by the sum of (I_(compensation)) and (I_(chime)) sothat the sum approximately equals (I_(camera)). In the standby or NOdoorbell mode, (I_(compensation)) may be greater than it would be whendoorbell system 1000 is in the doorbell operating mode.

In the doorbell operating mode, which starts at time, t1, camerasubsystem 1020 may continue to draw current (I_(camera)) and chimesubsystem 1050 may draw the sum of currents (I_(compensation)) and(I_(chime)). However, in the doorbell operating mode, (I_(chime)) may begreater than it was when operating in NO doorbell mode. As a result,current compensation network 1052 can dynamically reduce its(I_(compensation)) to ensure that the sum of (I_(compensation)) and(I_(chime)) is approximately equal to (I_(camera)).

FIG. 12 shows an illustrative current compensation network embodimentthat can be used in connection with a doorbell system according to anembodiment. In particular, FIG. 12 shows a portion of doorbell system1000 of FIG. 10, but includes additional details for currentcompensation network 1252, which can replace compensation network 1052.Current compensation network 1252 may include variable resistorcircuitry 1253 and control 1254. Variable resistor 1253 may be coupledto receive AC power from power source 1010 and is coupled to rectifier1055, control 1254, and the return path to power source 1010. Variableresistor 1253 can be operative to dynamically adjust its resistance tovary the amount of (I_(compensation)) current that it consumes based ona signal provided by control 1254. Control 1254 may control variableresistor 1253 based on whether the doorbell chime has been activated. Ifthe doorbell chime is not active, control 1254 may configure variableresistor 1253 to pull current (I_(compensation)) to compensate for thelack of current not being drawn by the doorbell chime. In other words,variable resistor 1253 may be configured to pull more current when thedoorbell chime is not active than when it is active. Thus, if thedoorbell chime is active, control 1254 may configure variable resistor1253 to pull current (I_(compensation)) that is less than the currentpulled when the doorbell chime is not active.

FIG. 13 shows an illustrative current compensation network embodimentthat can be used in connection with a doorbell system according to anembodiment. In particular, FIG. 13 shows a portion of doorbell system1000 of FIG. 10, but includes additional details for currentcompensation network 1352, which can replace compensation network 1052.Current compensation network 1352 may include resistor 1353, switch1354, and pulse width modulation (PWM) controller 1355. Switch 1354 isin parallel with resistor 1353. Controller 1355 can control how muchcurrent is consumed by compensation network 1352 by controlling the dutycycle of switch 1354. If the duty cycle is zero, then switch 1354 isOPEN and the amount of current consumed is controlled by the fullresistance value of resistor 1353. If duty cycle is one, then switch1354 is CLOSED and the current consumed by compensation network 1352 maybe maximized because the effective resistance of resistor 1353 isdecreased. Controller 155 can select also select a duty cycle betweenzero and one to control the amount of current consumed by compensationnetwork 1352. During operation, when the chime is inactive, switch 1354may be CLOSED or duty cycled ON (to increase current consumption by thecompensation network), and when the chime is active, switch 1354 maybeOPENED so that the current consumed by the compensation is current isminimal and the current is primarily provided to the chime.

FIG. 14 shows an illustrative current compensation network embodimentthat can be used in connection with a doorbell system according to anembodiment. In particular, FIG. 14 shows a portion of doorbell system1000 of FIG. 10, but includes additional details for currentcompensation network 1452, which can replace compensation network 1052.Current compensation network 1452 may include resistors 1453 and 1454,variable resistance component (VRC) 1455, amplifier 1456, and referencevoltage 1457. As shown, resistor 1453 is coupled to rectifier 1055, VRC1455, resistor 1454, and buck converter 1056. Resistor 1454 is coupledto rectifier 1055, buck converter 1056, and positive terminal ofamplifier 1456. Resistor 1458 is coupled to resistor 1454 and the returnto power source 1010. Amplifier 1456 is coupled to the output ofrectifier 1055. The output of amplifier 1456 is coupled to VRC 1455, thenegative terminal is coupled to reference voltage 1457. Amplifier 1456is ON when the voltage at its positive terminal is greater than thevoltage at its negative terminal, and amplifier 1456 is OFF when thevoltage at its positive terminal is less than the voltage at itsnegative terminal. VRC 1455 may function as a variable resistor orMOSFET. A MOSFET resistance can vary across a range—from (0′Ω to an“OPEN”. The voltage at the positive terminal may be greater than thereference voltage during standby or a non-button press mode, in whichcase, VRC 1455 is adjusted to increase the current consumed bycompensation circuit 1452. The voltage at the positive terminal may beless than the reference voltage during a button press event, in whichcase, VRC 1455 is adjusted to increase the current available forconsumption by the chime, and compensation circuit 1452 does not consumeany extra current.

For situations in which the systems discussed above collect informationabout users, the users may be provided with an opportunity to opt in/outof programs or features that may collect personal information (e.g.,information about a user's preferences or usage of a smart device). Inaddition, in some implementations, certain data may be anonymized in oneor more ways before it is stored or used, so that personallyidentifiable information is removed. For example, a user's identity maybe anonymized so that the personally identifiable information cannot bedetermined for or associated with the user, and so that user preferencesor user interactions are generalized (for example, generalized based onuser demographics) rather than associated with a particular user.

Although some of various drawings illustrate a number of logical stagesin a particular order, stages that are not order dependent may bereordered and other stages may be combined or broken out. While somereordering or other groupings are specifically mentioned, others will beobvious to those of ordinary skill in the art, so the ordering andgroupings presented herein are not an exhaustive list of alternatives.Moreover, it should be recognized that the stages could be implementedin hardware, firmware, software or any combination thereof.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific implementations. However, theillustrative discussions above are not intended to be exhaustive or tolimit the scope of the claims to the precise forms disclosed. Manymodifications and variations are possible in view of the aboveteachings. The implementations were chosen in order to best explain theprinciples underlying the claims and their practical applications, tothereby enable others skilled in the art to best use the implementationswith various modifications as are suited to the particular usescontemplated.

Reference will now be made in detail to implementations, examples ofwhich are illustrated in the accompanying drawings. In the followingdetailed description, numerous specific details are set forth in orderto provide a thorough understanding of the various describedimplementations. However, it will be apparent to one of ordinary skillin the art that the various described implementations may be practicedwithout these specific details. In other instances, well-known methods,procedures, components, mechanical structures, circuits, and networkshave not been described in detail so as not to unnecessarily obscureaspects of the implementations.

It will also be understood that, although the terms first, second, etc.are, in some instances, used herein to describe various elements, theseelements should not be limited by these terms. These terms are only usedto distinguish one element from another. For example, a first fastenerstructure can be termed a second fastener structure, and, similarly, asecond fastener structure can be termed a first fastener structure,without departing from the scope of the various describedimplementations. The first fastener structure and the second fastenerstructure are both fastener structures, but they are not the samefastener structure.

The terminology used in the description of the various describedimplementations herein is for the purpose of describing particularimplementations only and is not intended to be limiting. As used in thedescription of the various described implementations and the appendedclaims, the singular forms “a”, “an” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, components,structures and/or groups, but do not preclude the presence or additionof one or more other features, integers, steps, operations, elements,components, structures, and/or groups thereof

As used herein, the term “if’ is, optionally, construed to mean “when”or “upon” or “in response to determining” or “in response to detecting”or “in accordance with a determination that,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event]” or “in accordance with a determination that [astated condition or event] is detected,” depending on the context.

It is noted that the camera doorbells described herein are exemplary andare not intended to be limiting. For example, any dimensions, shapes,styles, and/or materials described herein are exemplary and are notintended to be limiting. Drawings are not to scale. For brevity,features or characters described in association with someimplementations may not necessarily be repeated or reiterated whendescribing other implementations. Even though it may not be explicitlydescribed therein, a feature or characteristic described in associationwith some implementations may be used by other implementations.

What is claimed is:
 1. A doorbell camera system, comprising: a cameradoorbell subsystem coupled to receive power from an alternating current(AC) power source, the camera doorbell subsystem comprising a doorbellbutton and a camera module; and a chime subsystem coupled to receivepower from the AC power source, the chime subsystem comprising: acurrent compensation network and chime driver circuitry operative to becoupled to a chime, wherein the current compensation network isoperative to dynamically adjust current consumption of the chimesubsystem and current consumption of the doorbell camera subsystemduring both a standby mode and a doorbell event mode.
 2. The doorbellcamera system of claim 1, wherein the doorbell camera system isconfigured to operate at the doorbell event mode when the doorbellbutton is pressed to activate the chime to produce a sound during abutton press event, and to operate at the standby mode when the doorbellbutton is not pressed.
 3. The doorbell camera system of claim 1, whereinthe camera doorbell subsystem includes a first processor, and the chimesubsystem includes a second processor operative to activate the chimedriver circuitry in response to a button press event of the doorbellbutton to activate the chime to produce a sound, and wherein the chimesubsystem further includes a battery, and the chime driver circuitryuses power supplied by the battery to activate the chime.
 4. Thedoorbell camera system of claim 3, wherein the first processor isoperative to communicate with the second processor, and wherein thefirst processor is operative to inform the second processor of thebutton press event.
 5. The doorbell camera system of claim 3, whereinthe first processor is operative to communicate with the secondprocessor via power line communications circuitry.
 6. The doorbellcamera system of claim 3, wherein the first processor is operative tocommunicate with the second processor via wireless communicationscircuitry.
 7. The doorbell camera system of claim 1, wherein the currentcompensation network comprises: variable resistor circuitry coupled tothe AC power source; and a switch that is coupled in series with thevariable resistor circuitry.
 8. The doorbell camera system of claim 1,wherein the current compensation network comprises: a comparator havinga first input, a second input, and an output; a first resistor coupledto a direct current (DC) power source; a switch coupled to the outputand the first resistor; a second resistor coupled to the DC power sourceand the first input; and a reference voltage coupled to the secondinput, wherein the comparator is operative to: turn ON the switch when avoltage signal on the first input exceeds the reference voltage; andturn OFF the switch when the voltage signal on the first input does notexceed the reference voltage.
 9. The doorbell camera system of claim 1,wherein during a doorbell button press event, the chime subsystemconsumes a first level of current, and wherein during the standby modein which there is no doorbell button press event, the chime subsystemconsumes a second level of current, wherein the second level of currentis greater than the first level of current.
 10. The doorbell camerasystem of claim 1, wherein the chime subsystem comprises the chime. 11.The doorbell camera system of claim 1, wherein: the camera doorbellsubsystem further includes a first power conditioning circuitry coupledto receive the AC power source; the camera module is coupled to receivepower from the first power conditioning circuitry; the chime subsystemfurther includes a battery, a second power conditioning circuitrycoupled to receive the AC power source, and a battery charging circuitrycoupled to receive power from the second power conditioning circuitryand configured to charge the battery with the received power; and thechime driver circuity is coupled to the battery charging circuitry andis operative to uses power supplied by the battery to activate thechime.
 12. The doorbell camera system of claim 11, wherein the cameradoorbell subsystem comprises first power line communications circuitrycoupled to a first processor of the camera doorbell subsystem, andwherein the chime subsystem comprises second power line communicationscircuitry coupled to a second processor of the chime subsystem.
 13. Thedoorbell camera system of claim 12, further comprising AC power sourceline, wherein the first and second power line communications circuitryare coupled to the AC power source line.
 14. The doorbell camera systemof claim 11, wherein the first power conditioning circuitry comprises arectifier and a buck converter, and wherein the second powerconditioning circuitry comprises a rectifier and a buck converter. 15.The doorbell camera system of claim 11, wherein: the camera doorbellsubsystem includes a first processor coupled to receive power from thefirst power conditioning circuitry; and the chime subsystem includes asecond processor that is coupled to receive power from the second powerconditioning circuitry and is operative to activate the chime drivercircuitry in response to a button press event of the doorbell button toactivate the chime to produce a sound at the doorbell event mode. 16.The doorbell camera system of claim 1, further comprising an alternatingcurrent (AC) power source connection node, wherein the camera doorbellsubsystem and the chime subsystem are coupled to the AC power sourceconnection node; wherein the chime subsystem further includes the chimeand a battery, and the battery supplies power to the chime when thechime is activated; and wherein the camera doorbell subsystem isoperative to instruct the chime subsystem to activate the chime inresponse to a button press of the doorbell button at the doorbell eventmode.
 17. The doorbell camera system of claim 16, wherein the cameradoorbell subsystem supplies uninterrupted DC power, derived from ACpower received via the AC power source connection node, to the cameramodule regardless of whether the chime is activated.
 18. The doorbellcamera system of claim 1, wherein the doorbell camera subsystemcommunicates with the chime subsystem via power line communicationscircuitry.
 19. The doorbell camera system of claim 1, wherein the cameradoorbell subsystem further includes a light emitting diode (LED)configured to illuminate through a peripheral edge of the doorbellbutton.
 20. The doorbell camera system of claim 1, wherein the cameramodule is driven by a first camera voltage, and the chime is driven by asecond chime voltage, and the current compensation network is configuredto control each of the first camera voltage and the second chime voltagewithin a respective voltage range.